-- Logs begin at Sun 2022-07-24 00:10:59 UTC, end at Sun 2022-07-24 01:21:04 UTC. -- Jul 24 00:51:49.372947 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 24 00:51:49.382488 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 24 00:51:49 2022] *** Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: nodename: dv-cinder-master-3-iscsi Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: machine: x86_64 Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: clock source: unix Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: pcre jit disabled Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: detected number of CPU cores: 4 Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: current working directory: / Jul 24 00:51:49.384490 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: detected binary path: /usr/bin/uwsgi-core Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: your processes number limit is 47680 Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: your memory page size is 4096 bytes Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: detected max file descriptor number: 2048 Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: lock engine: pthread robust mutexes Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: thunder lock: enabled Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 24 00:51:49.385172 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 24 00:51:49.428691 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: Python main interpreter initialized at 0x5609d33e99c0 Jul 24 00:51:49.428691 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: python threads support enabled Jul 24 00:51:49.428691 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: your server socket listen backlog is limited to 100 connections Jul 24 00:51:49.428691 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: your mercy for graceful operations on workers is 90 seconds Jul 24 00:51:49.429033 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: mapped 403077 bytes (393 KB) for 2 cores Jul 24 00:51:49.429110 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: *** Operational MODE: preforking *** Jul 24 00:51:49.429365 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: *** uWSGI is running in multiple interpreter mode *** Jul 24 00:51:49.429365 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: spawned uWSGI master process (pid: 103246) Jul 24 00:51:49.429565 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 24 00:51:49.430638 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: spawned uWSGI worker 1 (pid: 103247, cores: 1) Jul 24 00:51:49.430638 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: spawned uWSGI worker 2 (pid: 103248, cores: 1) Jul 24 00:51:49.430638 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 24 00:51:50.091759 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 24 00:51:50.092361 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 24 00:51:51.222797 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:51:51.222797 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: from cryptography.utils import int_from_bytes Jul 24 00:51:51.223390 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:51:51.223390 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: from cryptography.utils import int_from_bytes Jul 24 00:51:51.223536 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:51:51.223536 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: from cryptography.utils import int_from_bytes Jul 24 00:51:51.223536 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:51:51.223536 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: from cryptography.utils import int_from_bytes Jul 24 00:51:53.733159 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 24 00:51:53.733776 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 24 00:51:53.733874 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 24 00:51:53.734012 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 24 00:51:53.734381 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 24 00:51:53.734468 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 24 00:51:53.734557 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 24 00:51:53.734806 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 24 00:51:53.734898 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.735167 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.735270 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 24 00:51:53.735430 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 24 00:51:53.735517 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.735809 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.735901 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.736140 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.736233 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.736455 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.736551 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.736779 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.736873 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.737102 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.737188 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.737417 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.737504 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.737688 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.737775 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738010 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738106 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738340 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738428 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738618 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738727 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.738978 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.739088 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.739295 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.739382 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.739620 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.739756 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.739929 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.740067 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.740256 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.740384 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.740567 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.740708 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.740915 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.741137 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.741276 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.742360 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.742578 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.742811 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.743026 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.743232 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.743411 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.743569 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.743794 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.743952 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.744194 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.744383 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.744586 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.744738 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.744916 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.745130 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.745325 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.745518 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.745676 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.745898 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.746094 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.746279 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.746989 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.747141 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.747361 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.747556 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.747707 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.747859 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748017 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748183 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748318 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748442 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748608 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748608 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.748861 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749006 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749200 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749361 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749495 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749632 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749765 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.749898 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750022 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750146 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750283 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750408 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750531 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750686 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750841 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.750999 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751335 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751470 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751595 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751718 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751853 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.751995 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752127 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752258 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752383 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752513 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752513 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752776 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.752901 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753025 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753025 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753299 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753461 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753598 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753722 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753844 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.753844 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754080 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754211 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754336 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754476 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754607 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754607 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.754894 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755021 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755145 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755267 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755389 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755520 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755643 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755772 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.755901 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756032 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756164 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756297 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756421 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756542 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756670 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756804 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.756926 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757057 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757180 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757309 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757435 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757558 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757679 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757815 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.757938 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758060 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758188 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758312 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758452 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.18 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758576 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758706 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.18 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758871 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.758997 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.18 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.759121 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.18 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.759340 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.759471 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.759604 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.759772 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.759900 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.760067 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.760209 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.760383 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.760519 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.760696 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.760829 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761011 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761147 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761324 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761452 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761639 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761766 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.761955 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.762689 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.762854 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.762979 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763101 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763223 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763397 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763570 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763686 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763824 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.763984 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.764120 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.764298 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.764427 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.764611 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.764740 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.764923 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765050 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765230 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765366 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765543 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765678 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765856 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.765984 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.766168 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.766303 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.766480 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.766614 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.766810 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.766955 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.767124 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.767252 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.767434 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.767559 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.767741 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.767953 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768081 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768216 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768340 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768469 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768644 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768787 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.768966 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.769099 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.769278 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.769486 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.769614 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.769749 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.769888 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770014 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770192 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770332 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770500 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770636 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770834 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.770966 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.771161 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.771287 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.771471 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.771613 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.771783 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.771914 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.772095 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.772229 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.772405 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.772542 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.772724 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.772852 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773034 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773160 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773337 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773474 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773650 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773788 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.773954 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.774081 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.774268 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.774398 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.774575 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.774708 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.774913 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775043 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775222 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775357 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775528 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775655 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775834 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.775968 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.776140 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.776275 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.776447 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.776580 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.776750 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.776885 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.777057 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.777195 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.777362 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.777492 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.777751 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.777890 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.778081 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.778241 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.778393 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.778562 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.778718 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.778903 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.779086 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.779214 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.779423 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.779628 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:51:53.779758 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.779893 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.780077 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.780205 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.780414 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.780551 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.780751 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.780890 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.781077 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.781205 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.781396 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.781517 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.781717 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.781846 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.782048 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.782184 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.782367 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.782496 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.782690 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.782846 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783033 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783165 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783350 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783486 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783671 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783799 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.783998 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.784151 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.784318 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.784446 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.784640 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.784771 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.784959 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.785087 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.785273 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.785408 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.785595 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.785738 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.785912 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.786049 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.786234 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.786368 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.786547 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.786676 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.786900 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.787037 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.787217 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.787359 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.787559 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.787687 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.787891 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788019 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788210 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788347 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788527 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788654 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788843 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.788971 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.789157 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.789358 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.789486 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.789625 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.789824 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.789962 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.790095 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.790229 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.790418 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.790560 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.790736 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.790897 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.791082 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.791205 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.791400 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.791537 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.791718 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.791847 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.792035 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.792164 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.792351 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.792478 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.792671 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.792800 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793004 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793140 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793348 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793477 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793673 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793800 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.793993 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.794122 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.794318 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.794447 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.794640 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.794826 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.794985 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.795126 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.795307 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.795435 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.795620 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.795762 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.795934 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.796063 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.796247 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.796376 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.796565 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.796702 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.796876 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797005 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797203 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797331 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797538 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797674 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797850 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.797977 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.798169 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.798304 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.798485 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.798613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.798819 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.798949 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.799136 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.799265 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.799456 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.799593 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.799769 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.799983 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800112 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800245 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800379 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800503 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800683 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800811 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.800993 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.801119 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.801313 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.801441 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.801622 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.801750 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.801933 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.802071 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.802245 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.802374 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.802556 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.802685 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.802902 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803042 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803218 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803337 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803533 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803660 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803850 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.803992 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.804169 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.804296 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.804488 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.804624 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.804805 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.804947 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.805127 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.805264 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.805450 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.805578 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.805779 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.805916 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.806088 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.806223 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.806418 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.806546 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.806730 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.806894 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807071 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807200 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807381 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807509 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807690 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807818 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.807998 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.808125 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.808566 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.808694 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.809390 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.809579 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.809721 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.809890 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.810101 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.810279 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.810410 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.810596 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.810735 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.810925 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.811063 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.811231 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.811361 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.811559 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.811696 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.811871 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812000 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812179 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812307 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812487 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812614 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812793 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.812921 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.813139 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.813267 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.813446 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.813574 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.813793 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.813934 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.814129 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.814260 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.814450 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.814584 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.814798 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.814947 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.815108 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.815243 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.815425 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.815553 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.815741 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.815883 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.816056 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.816184 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.816372 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.816500 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.816689 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.816827 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817007 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817135 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817324 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817457 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817641 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817777 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.817955 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.818084 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.818274 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.818411 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.818589 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.818716 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.818928 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.819057 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.819238 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.819366 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.819567 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.819696 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.819897 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820025 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820230 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820366 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820531 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820661 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820851 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.820986 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.821167 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.821294 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.821483 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.821611 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.821796 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.821916 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.822111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.822249 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.822425 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.822559 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.822742 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.822898 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.823084 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.823229 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.823398 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.823518 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.823717 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.823844 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.824031 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.824166 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.824344 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.824518 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.825276 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.825407 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.825537 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.825714 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.825863 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.825989 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.826153 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.826291 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.826488 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.826632 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.826854 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.826986 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.827173 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.827302 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.827493 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.827624 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.827821 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.827949 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.828138 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.828274 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.828454 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.828585 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.828772 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.828902 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.829083 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.829217 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.829398 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.829526 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.829717 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.829846 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830033 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830167 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830385 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830515 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830641 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830783 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.830984 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.831131 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.831371 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.831855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.831982 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.832184 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.832312 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.832501 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.832631 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.832824 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.832959 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.833158 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.833374 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.833512 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.833628 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.833796 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.833925 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.834131 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.834260 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.834435 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.834533 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.834749 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.834887 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.835099 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.835207 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.835416 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.835503 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.835727 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.835814 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.836039 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.836134 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.836347 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.836433 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.836697 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.836783 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837015 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837109 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837327 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837421 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837650 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837741 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.837975 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.838071 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.838291 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.838383 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.838608 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.838693 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.838951 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.839039 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.839270 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.839367 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.839601 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.839695 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.839921 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.840015 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.840334 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.840478 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.840669 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.840763 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.840987 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.841075 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.841307 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.841395 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.841626 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.841713 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.841955 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.842060 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.842273 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.842366 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.842586 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.842691 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.842926 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843014 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843237 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843324 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843551 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843665 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843874 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.843978 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.844449 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.844536 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.844787 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.844880 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.845108 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.845193 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.845430 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.845514 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.845753 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.845846 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.846075 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.846161 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.846410 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.846494 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.846741 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.846845 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.847112 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.847215 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.847423 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.847519 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.847736 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.847821 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848044 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848141 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848351 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848443 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848664 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848749 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.848971 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849064 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849280 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849365 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849592 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849904 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.849989 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.850209 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.850293 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.850514 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.850598 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.850846 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.850939 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.851157 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.851244 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.851475 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.851560 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.851781 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.851868 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.852097 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.852191 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.852405 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.852498 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.852731 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.852825 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853042 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853127 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853347 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853445 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853653 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853739 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.853957 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.854043 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.854302 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.854398 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.854610 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.854702 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.854973 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.855060 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.855307 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.855406 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.855652 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.855738 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.855991 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.856080 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.856302 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.856399 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.856629 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.856714 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.856936 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857020 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857243 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857336 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857549 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857634 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857862 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.857948 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.858172 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.858258 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.858476 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.858562 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.858815 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.858905 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.859124 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.859212 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.859441 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.859527 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.859751 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.859835 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860057 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860145 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860367 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860453 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860673 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860771 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.860989 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861087 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861300 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861394 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861607 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861700 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861911 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.861997 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.862223 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.862308 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.862535 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.862632 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.862866 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.862955 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.863181 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.863268 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.863487 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.863580 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.863803 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.863888 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.864113 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.864203 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.864419 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.864776 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.864776 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.864928 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865007 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865085 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865321 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865416 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865626 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865711 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.865949 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866033 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866264 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866350 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866572 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866663 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866905 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.866997 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.867223 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.867310 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.867529 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.867615 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.867835 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.867929 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.868148 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.868271 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.868474 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.868559 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.868793 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.868886 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.869099 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.869184 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.869409 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.869508 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.869716 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.869809 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870021 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870116 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870329 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870421 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870638 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870834 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.870972 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.871071 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.871276 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.871865 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.872123 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.872282 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.872434 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.872614 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.872744 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.872932 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.873073 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.873267 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.873387 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.873588 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.873696 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.873903 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.874002 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.874218 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.874315 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.874529 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.874652 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.874905 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875076 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875220 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875309 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875585 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875672 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875901 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.875989 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.876226 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.876311 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.876545 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.876631 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.876861 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.876955 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.877178 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.877264 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.877488 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.877574 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.877796 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.877883 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.878112 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.878203 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.878426 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.878510 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.878732 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.878842 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.879067 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.879154 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.879396 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.879482 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.879715 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.879803 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.880032 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.880125 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.880358 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.880458 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.880683 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.880769 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881009 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881095 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881322 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881432 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881636 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881722 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.881952 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.882046 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.882267 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.882360 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.882579 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.882663 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.882914 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883002 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883230 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883328 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883544 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883637 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883860 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.883953 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.884175 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.884268 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.884495 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.884588 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.884804 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.884888 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.885114 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.885201 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.885424 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.885509 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.885734 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.885820 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.886041 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.886127 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.886365 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.886466 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.886675 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.886762 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.887077 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.887748 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.887928 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888015 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888244 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888340 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888555 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888649 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888864 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.888988 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.889300 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.889397 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.889616 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.889703 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.889928 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890015 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890236 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890335 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890549 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890635 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890889 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.890977 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.891228 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.891368 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.891522 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.891610 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.891829 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.891944 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.892248 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.892336 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.892562 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.892655 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.892878 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.892962 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.893185 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.893271 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.893491 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.893577 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.893800 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.893886 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.894114 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.894203 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.894426 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.894513 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.894734 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.894880 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895069 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895157 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895378 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895465 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895691 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895789 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.895998 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.896083 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.896322 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.896408 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.896731 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.896829 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.897043 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.897129 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.897350 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.897467 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.897773 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.897868 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.898089 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.898174 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.898403 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.898489 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.898730 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.898837 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.899081 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.899166 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.899391 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.899478 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.899712 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.899809 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900029 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900131 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900341 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900427 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900671 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900765 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.900986 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.901073 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.901303 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.901388 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.901613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.901699 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.901925 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.902012 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.902256 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.902343 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.902575 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.902716 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.903001 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.903099 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.903334 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.903509 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.903755 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.903842 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.904090 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.904187 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.904411 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.904498 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.904739 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.904834 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905073 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905161 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905381 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905468 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905689 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905784 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.905999 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.906093 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.906310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.906403 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.906623 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.906719 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.906963 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907051 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907278 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907375 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907593 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907687 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907907 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.907993 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.908217 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.908303 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.908525 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.908612 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.908836 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.908923 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.909149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.909240 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.909462 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.909554 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.909783 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.909869 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.910104 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.910196 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.910427 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.910515 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.910740 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.910862 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.911078 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.911163 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.911397 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.911485 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.911709 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.911796 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912016 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912109 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912323 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912419 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912631 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912724 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.912969 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.913109 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.913276 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.913363 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.913604 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.913691 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.913934 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.914029 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.914258 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.914353 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.914570 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.914662 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.914938 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915029 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915262 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915358 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915584 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915670 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915910 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.915998 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.916222 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.916308 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.916531 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.916627 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.916837 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.916924 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.917144 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.917236 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.917451 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.917538 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.917756 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.917844 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.918063 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.918148 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.918411 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.918505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.918720 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.918864 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.919054 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.919166 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.919363 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.919496 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.919698 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.919809 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920008 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920138 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920335 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920439 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920651 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920748 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.920960 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.921047 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.921271 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.921367 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.921580 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.921668 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.921929 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.922016 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.922239 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.922334 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.922577 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.922663 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.922931 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923023 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923261 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923351 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923573 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923659 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923900 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.923993 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.924229 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.924322 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.924551 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.924681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.924870 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.924965 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.925218 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.925310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.925508 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.925594 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.925826 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.925912 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.926145 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.926232 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.926459 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.926547 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.926782 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.926882 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.927116 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.927205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.927436 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.927528 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.927754 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.927965 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.928288 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.928383 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.928615 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.928707 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.928936 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929032 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929256 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929352 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929578 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929714 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929899 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.929994 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.930215 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.930301 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.930534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.930620 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.930875 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.930976 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.931193 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.931282 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.931511 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.931597 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.931840 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.931935 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.932168 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.932254 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.932502 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.932597 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.932823 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.932909 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.933391 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.933528 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.933720 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.933815 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.934557 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.934651 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.934955 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.935086 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.935275 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.935365 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.935590 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.935686 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.935911 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936002 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936233 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936328 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936556 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936644 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936876 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.936963 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.937194 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.937287 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.937515 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.937601 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.937835 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.937969 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.938155 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.938300 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.938473 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.938624 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.938818 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.938978 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.939138 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.939302 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.939468 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.939626 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.939814 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.939967 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.940143 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.940315 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.940467 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.940645 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.940791 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.940970 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.941117 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.941293 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.941620 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.941705 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.941953 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.942154 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.942295 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.942485 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.942628 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.942927 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.943030 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.943271 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.943359 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.943606 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.943699 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.943931 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944026 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944243 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944329 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944562 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944656 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944883 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.944969 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.945196 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.945283 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.945507 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.945592 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.945825 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.945912 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.946140 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.946238 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.946454 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.946541 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.946808 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.946898 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.947114 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.947199 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.947440 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.947653 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.947749 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.948008 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.948284 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.948371 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.948640 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.948739 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.948967 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.949052 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.949292 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.949379 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.950207 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.950314 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.950543 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.950646 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.950878 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.950990 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.951195 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.951298 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.951515 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.951602 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.951830 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.951915 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.952142 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.952231 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.952455 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.952542 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.952762 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.952848 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.953074 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.953165 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.953387 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.953475 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.953700 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.953785 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954008 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954103 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954323 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954417 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954635 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954730 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.954974 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955062 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955286 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955383 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955597 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955691 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955907 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.955992 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.956217 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.956306 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.956525 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.956624 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.956833 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.956920 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.957143 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.957228 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.957450 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.957546 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.957765 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.957852 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.958073 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.958160 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.958387 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.958481 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.958694 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.958798 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.959036 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.959122 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.959347 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.959445 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.959662 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.959750 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960004 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960090 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960315 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960404 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960637 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960733 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.960946 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961032 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961270 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961358 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961585 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961671 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961900 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.961989 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.962219 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.962307 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.962538 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.962633 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.962918 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.963120 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.963242 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.963461 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.963782 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.963870 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.964117 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.964211 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.964441 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.964526 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.964764 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.964850 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.965076 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.965202 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.965711 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.965811 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.965892 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.965971 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.966126 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.966228 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.966445 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.966533 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.966764 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.966862 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.967108 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.967196 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.967422 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.967509 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.967736 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.967824 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968050 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968138 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968363 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968449 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968676 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968763 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.968988 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.969076 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.969300 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.969427 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.969612 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.969708 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.969923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970016 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970242 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970327 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970554 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970689 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970908 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.970998 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.971229 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.971315 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.971551 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.971651 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.971874 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.971970 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.972192 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.972280 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.972508 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.972604 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.972820 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.972906 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.973132 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.973217 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.973444 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.973531 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.973759 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.973847 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.974078 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.974164 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.974396 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.974491 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.974712 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.974844 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.975050 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.975173 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.975459 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.975547 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.975776 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.975861 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.976089 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.976183 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.976416 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.976501 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.976732 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.976819 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.977047 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.977141 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.977375 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.977463 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.977689 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.977775 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.978016 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.978111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.978334 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.978421 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.978668 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.978763 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979032 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979120 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979345 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979433 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979657 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979746 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.979969 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.980057 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.980285 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.980372 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.980598 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.980687 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.981443 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.981545 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.981750 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.981838 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.982060 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.982166 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.982370 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.982455 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.982678 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.982764 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983016 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983116 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983371 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983465 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983631 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983718 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.983938 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984079 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984242 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984329 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984552 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984644 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984860 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.984958 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.985165 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.985288 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.985513 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.985600 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.985829 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.985973 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.986141 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.986236 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.986449 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.986590 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.986762 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.986893 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.987093 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.987180 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.987400 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.987493 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.987706 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.987793 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988017 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988141 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988325 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988414 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988661 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988750 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.988987 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.989074 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.989304 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.989398 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.989627 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.989713 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.989942 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.990031 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.990269 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.990364 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.990581 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.990678 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.990925 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991026 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991243 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991340 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991560 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991648 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991877 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.991963 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.992191 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.992350 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.992510 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.992598 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.992825 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.992910 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.993138 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.993232 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.993451 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.993536 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.993768 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.993854 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.994086 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.994181 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.994403 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.994489 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.994734 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.994841 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.995071 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.995158 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.995394 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.995486 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.995716 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.995812 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.996034 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.996122 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.996373 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.997039 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.997235 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.997331 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.997555 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.997794 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.997881 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.998119 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.998206 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.998443 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.998534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.998763 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.998879 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.999105 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.999202 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.999421 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.999515 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.999742 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:53.999828 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.000059 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.000157 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.000382 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.000475 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.000708 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.000795 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001021 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001115 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001353 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001440 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001767 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.001999 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.002094 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.002331 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.002424 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.002659 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.002752 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003024 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003114 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003343 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003430 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003663 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003749 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.003976 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.004065 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.004227 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 24 00:51:54.004315 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.004622 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.004936 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:51:54.005188 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 24 00:51:54.067541 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO dbcounter [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Registered counter for database nova_api Jul 24 00:51:54.068455 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO dbcounter [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Registered counter for database nova_api Jul 24 00:51:54.180969 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_db.sqlalchemy.engines [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.182700 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_db.sqlalchemy.engines [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103247) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.187949 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writer thread running {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.189418 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writer thread running {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.244617 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.context [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a1472727-7c57-4e7e-873f-6498708782b3(cell1) {{(pid=103248) load_cells /opt/stack/nova/nova/context.py:464}} Jul 24 00:51:54.245706 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.context [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a1472727-7c57-4e7e-873f-6498708782b3(cell1) {{(pid=103247) load_cells /opt/stack/nova/nova/context.py:464}} Jul 24 00:51:54.248566 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:51:54.249689 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_concurrency.lockutils [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103247) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:51:54.250132 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:51:54.250987 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:51:54.251870 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_concurrency.lockutils [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103247) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:51:54.252976 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_concurrency.lockutils [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103247) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:51:54.253583 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:51:54.255614 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO dbcounter [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Registered counter for database nova_cell0 Jul 24 00:51:54.256467 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_concurrency.lockutils [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=103247) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:51:54.259516 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO dbcounter [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Registered counter for database nova_cell1 Jul 24 00:51:54.259632 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO dbcounter [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Registered counter for database nova_cell0 Jul 24 00:51:54.264395 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO dbcounter [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Registered counter for database nova_cell1 Jul 24 00:51:54.270903 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_db.sqlalchemy.engines [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.271966 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_db.sqlalchemy.engines [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.275748 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_db.sqlalchemy.engines [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103247) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.276646 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_db.sqlalchemy.engines [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103247) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.280642 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writer thread running {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.281971 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writer thread running {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.285010 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writer thread running {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.286600 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writer thread running {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.447325 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO dbcounter [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Registered counter for database nova_cell0 Jul 24 00:51:54.457538 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG oslo_db.sqlalchemy.engines [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103247) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.458849 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO dbcounter [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Registered counter for database nova_cell0 Jul 24 00:51:54.462116 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writer thread running {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.468397 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_db.sqlalchemy.engines [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:51:54.473351 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writer thread running {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:51:54.816211 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: WARNING keystonemiddleware.auth_token [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:51:54.824098 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: WARNING keystonemiddleware.auth_token [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:51:54.844740 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: WARNING keystonemiddleware.auth_token [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:51:54.852295 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: WARNING keystonemiddleware.auth_token [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:51:55.175943 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: WARNING keystonemiddleware.auth_token [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:51:55.182519 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: WARNING keystonemiddleware.auth_token [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:51:55.188742 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5609d33e99c0 pid: 103248 (default app) Jul 24 00:51:55.190829 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:51:55.192024 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] 10.222.0.18 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001762 Jul 24 00:51:55.192435 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 788 bytes} [Sun Jul 24 00:51:55 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:51:55.201600 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: WARNING keystonemiddleware.auth_token [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:51:55.207635 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: WARNING keystonemiddleware.auth_token [None req-15250a92-062f-4b71-ac55-3520fad1d64a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:51:55.212898 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5609d33e99c0 pid: 103247 (default app) Jul 24 00:53:11.014481 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:11.016115 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002050 Jul 24 00:53:11.016685 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:11.033719 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writing DB stats nova_api:SELECT=3 {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:11.035280 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writing DB stats nova_cell0:SELECT=3 {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:11.036196 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writing DB stats nova_cell1:SELECT=3 {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:11.037206 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:11.902486 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-1cb4a530-4946-4fd6-9069-d8d995c6690e admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:11.934137 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-1cb4a530-4946-4fd6-9069-d8d995c6690e admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.910500 Jul 24 00:53:11.934914 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 1/3] 10.222.0.18 () {60 vars in 1291 bytes} [Sun Jul 24 00:53:11 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 912 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:14.187566 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:14.188676 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-94fa828c-4b72-4597-b521-4aaa7d15aeec None None] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001529 Jul 24 00:53:14.189363 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 3/4] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:14.658756 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-f6557cdf-d4c2-4d2d-b024-47789918af92 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 1, "id": "c1", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "name": "cirros256"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:14.677541 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:53:14.677541 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: warnings.warn(msg) Jul 24 00:53:14.706444 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-f6557cdf-d4c2-4d2d-b024-47789918af92 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.511302 Jul 24 00:53:14.707035 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 2/5] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:14 2022] POST /compute/v2.1/flavors => generated 407 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:53:14.721502 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_api:SELECT=3 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:14.722710 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_cell1:SELECT=3 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:14.723517 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_cell0:SELECT=3 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:14.724598 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:14.934887 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-bfbd1603-0d19-4625-9096-531f55dc3aab admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:14.937315 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:53:14.937315 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: warnings.warn(msg) Jul 24 00:53:14.999257 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-bfbd1603-0d19-4625-9096-531f55dc3aab admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.283154 Jul 24 00:53:15.000120 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 4/6] 10.222.0.18 () {64 vars in 1335 bytes} [Sun Jul 24 00:53:14 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:17.262353 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-f6557cdf-d4c2-4d2d-b024-47789918af92 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:17.263555 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-f6557cdf-d4c2-4d2d-b024-47789918af92 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001657 Jul 24 00:53:17.264060 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 3/7] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:17 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 00:53:17.941721 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-4f568426-a7af-4799-8a86-1d1a39027371 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 512, "name": "ds512M", "disk": 5, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d1", "os-flavor-access:is_public": true, "vcpus": 1, "rxtx_factor": 1.0}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:17.962927 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:53:17.962927 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: warnings.warn(msg) Jul 24 00:53:17.983620 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-4f568426-a7af-4799-8a86-1d1a39027371 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.713798 Jul 24 00:53:17.984737 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 5/8] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:17 2022] POST /compute/v2.1/flavors => generated 404 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 24 00:53:17.996613 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-e8ad3928-bd9f-4a6c-a461-11f37aeced3a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:17.998488 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:53:17.998488 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: warnings.warn(msg) Jul 24 00:53:18.034379 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-e8ad3928-bd9f-4a6c-a461-11f37aeced3a admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.041537 Jul 24 00:53:18.035269 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 4/9] 10.222.0.18 () {64 vars in 1335 bytes} [Sun Jul 24 00:53:17 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:20.263756 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-4f568426-a7af-4799-8a86-1d1a39027371 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:20.264652 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-4f568426-a7af-4799-8a86-1d1a39027371 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001483 Jul 24 00:53:20.281559 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:20 2022] GET /compute/v2.1 => generated 388 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:20.669401 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-d6edc3a9-0d93-4a05-b553-d7d6980ad4e9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds1G", "swap": 0, "rxtx_factor": 1.0, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d2", "vcpus": 1, "ram": 1024, "os-flavor-access:is_public": true}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:20.688012 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-d6edc3a9-0d93-4a05-b553-d7d6980ad4e9 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.417003 Jul 24 00:53:20.688556 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 5/11] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:20 2022] POST /compute/v2.1/flavors => generated 404 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:53:20.701149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-0de2d9ac-aa8c-4e45-b5eb-cca68a301c5d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:20.723699 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-0de2d9ac-aa8c-4e45-b5eb-cca68a301c5d admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026149 Jul 24 00:53:20.724259 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 7/12] 10.222.0.18 () {64 vars in 1335 bytes} [Sun Jul 24 00:53:20 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:22.940911 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-d6edc3a9-0d93-4a05-b553-d7d6980ad4e9 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:22.942087 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-d6edc3a9-0d93-4a05-b553-d7d6980ad4e9 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001607 Jul 24 00:53:22.942588 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 6/13] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:22 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:23.367318 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-2fadb78b-9c35-4f6e-97a1-f35ebd48fb05 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "disk": 10, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "id": "d3", "name": "ds2G", "os-flavor-access:is_public": true, "swap": 0}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:23.384819 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-2fadb78b-9c35-4f6e-97a1-f35ebd48fb05 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.436137 Jul 24 00:53:23.385298 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 8/14] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:22 2022] POST /compute/v2.1/flavors => generated 404 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 24 00:53:23.397280 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-ef731934-24ea-4cd9-9516-f2001e9551ca admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:23.430227 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-ef731934-24ea-4cd9-9516-f2001e9551ca admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035230 Jul 24 00:53:23.431267 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 7/15] 10.222.0.18 () {64 vars in 1335 bytes} [Sun Jul 24 00:53:23 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 00:53:25.691961 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-2fadb78b-9c35-4f6e-97a1-f35ebd48fb05 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:25.692894 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-2fadb78b-9c35-4f6e-97a1-f35ebd48fb05 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Jul 24 00:53:25.693496 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 9/16] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:26.118861 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-921f0669-fb73-4381-9db1-1a348787c15e admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds4G", "ram": 4096, "id": "d4", "os-flavor-access:is_public": true, "swap": 0, "rxtx_factor": 1.0, "vcpus": 4}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:26.138189 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-921f0669-fb73-4381-9db1-1a348787c15e admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.438354 Jul 24 00:53:26.138850 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 8/17] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:25 2022] POST /compute/v2.1/flavors => generated 404 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:53:26.151705 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-1583acbe-eeb5-4163-8ddb-1c2a14263081 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:26.178046 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-1583acbe-eeb5-4163-8ddb-1c2a14263081 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030021 Jul 24 00:53:26.187432 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 10/18] 10.222.0.18 () {64 vars in 1335 bytes} [Sun Jul 24 00:53:26 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 00:53:28.440277 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-921f0669-fb73-4381-9db1-1a348787c15e admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:28.441364 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-921f0669-fb73-4381-9db1-1a348787c15e admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001518 Jul 24 00:53:28.441767 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 9/19] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 00:53:28.921708 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-1f9bb612-09ab-417a-8187-6da450541735 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:28.949688 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-1f9bb612-09ab-417a-8187-6da450541735 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.500205 Jul 24 00:53:28.950340 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 11/20] 10.222.0.18 () {60 vars in 1291 bytes} [Sun Jul 24 00:53:28 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:53:31.311317 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-921f0669-fb73-4381-9db1-1a348787c15e admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:31.312532 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-921f0669-fb73-4381-9db1-1a348787c15e admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001738 Jul 24 00:53:31.312992 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 10/21] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:31 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:31.780968 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-a7870423-2809-4a01-88a3-fb53d34004ea admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "swap": 0, "name": "m1.tiny", "ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 1, "id": "1", "rxtx_factor": 1.0}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:31.799195 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-a7870423-2809-4a01-88a3-fb53d34004ea admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.479400 Jul 24 00:53:31.799754 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 12/22] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:31 2022] POST /compute/v2.1/flavors => generated 402 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 24 00:53:31.812580 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-c25747ac-b550-430a-8fc1-6ad51155aacf admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:31.838347 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-c25747ac-b550-430a-8fc1-6ad51155aacf admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029273 Jul 24 00:53:31.839141 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 11/23] 10.222.0.18 () {64 vars in 1332 bytes} [Sun Jul 24 00:53:31 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:34.379524 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-a7870423-2809-4a01-88a3-fb53d34004ea admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:34.380553 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-a7870423-2809-4a01-88a3-fb53d34004ea admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001513 Jul 24 00:53:34.380959 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:34.778980 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-c44d7ec5-cc52-4d38-abfe-d002f6f1f0aa admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.small", "os-flavor-access:is_public": true, "id": "2", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "swap": 0, "ram": 2048, "rxtx_factor": 1.0, "disk": 20}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:34.796093 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-c44d7ec5-cc52-4d38-abfe-d002f6f1f0aa admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.409296 Jul 24 00:53:34.796864 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 12/25] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:34 2022] POST /compute/v2.1/flavors => generated 405 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 24 00:53:34.809175 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-3eeb2aa9-5d12-4cb8-af85-53fe0a9c803c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:34.831882 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-3eeb2aa9-5d12-4cb8-af85-53fe0a9c803c admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026864 Jul 24 00:53:34.832451 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 14/26] 10.222.0.18 () {64 vars in 1332 bytes} [Sun Jul 24 00:53:34 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 00:53:37.088807 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-c44d7ec5-cc52-4d38-abfe-d002f6f1f0aa admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:37.089813 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-c44d7ec5-cc52-4d38-abfe-d002f6f1f0aa admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 Jul 24 00:53:37.090216 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 13/27] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:37.451793 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-59fb2b54-4176-4746-97d1-819d26374293 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 40, "swap": 0, "ram": 4096, "os-flavor-access:is_public": true, "name": "m1.medium", "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "3"}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:37.470846 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-59fb2b54-4176-4746-97d1-819d26374293 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.374588 Jul 24 00:53:37.479139 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 15/28] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:37 2022] POST /compute/v2.1/flavors => generated 406 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:53:37.484043 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-1a6328e1-ef1b-4071-8987-5809f00693ca admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:37.520119 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-1a6328e1-ef1b-4071-8987-5809f00693ca admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039574 Jul 24 00:53:37.520850 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 14/29] 10.222.0.18 () {64 vars in 1332 bytes} [Sun Jul 24 00:53:37 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:39.854476 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-59fb2b54-4176-4746-97d1-819d26374293 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:39.856229 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-59fb2b54-4176-4746-97d1-819d26374293 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002199 Jul 24 00:53:39.857233 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 16/30] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:39 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:40.283445 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-07f045ba-1de8-4a35-9edd-f08b0f0e8843 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 8192, "vcpus": 4, "name": "m1.large", "disk": 80, "id": "4"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:40.302432 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-07f045ba-1de8-4a35-9edd-f08b0f0e8843 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.439769 Jul 24 00:53:40.303344 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 15/31] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:39 2022] POST /compute/v2.1/flavors => generated 405 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:53:40.315612 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-3fd0f47b-fe2d-4034-8624-c4ce71e59bf7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:40.339849 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-3fd0f47b-fe2d-4034-8624-c4ce71e59bf7 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027792 Jul 24 00:53:40.340409 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 17/32] 10.222.0.18 () {64 vars in 1332 bytes} [Sun Jul 24 00:53:40 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:53:42.629875 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-07f045ba-1de8-4a35-9edd-f08b0f0e8843 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:53:42.630857 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-07f045ba-1de8-4a35-9edd-f08b0f0e8843 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001510 Jul 24 00:53:42.631537 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 16/33] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:53:42 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:53:43.083647 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-55185797-9b98-4a5b-8bc2-2746611adf5c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.xlarge", "rxtx_factor": 1.0, "ram": 16384, "id": "5", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 8, "swap": 0, "disk": 160}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:43.105950 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-55185797-9b98-4a5b-8bc2-2746611adf5c admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.468882 Jul 24 00:53:43.106642 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 18/34] 10.222.0.18 () {64 vars in 1282 bytes} [Sun Jul 24 00:53:42 2022] POST /compute/v2.1/flavors => generated 408 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:53:43.128492 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-4884f3bb-4b15-4456-a9aa-a7c7a435ebc8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:53:43.133094 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=9,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:53:43.154533 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-4884f3bb-4b15-4456-a9aa-a7c7a435ebc8 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035086 Jul 24 00:53:43.164114 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 17/35] 10.222.0.18 () {64 vars in 1332 bytes} [Sun Jul 24 00:53:43 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:54:02.786140 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-55185797-9b98-4a5b-8bc2-2746611adf5c admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:02.787184 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-55185797-9b98-4a5b-8bc2-2746611adf5c admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001439 Jul 24 00:54:02.787581 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 19/36] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:54:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:03.170297 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-55e00ac1-952b-477d-ac1b-bffedbc2517d admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:03.187256 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-55e00ac1-952b-477d-ac1b-bffedbc2517d admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.393255 Jul 24 00:54:03.187786 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 18/37] 10.222.0.18 () {60 vars in 1291 bytes} [Sun Jul 24 00:54:02 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:54:04.247019 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-55185797-9b98-4a5b-8bc2-2746611adf5c admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:04.248030 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-55185797-9b98-4a5b-8bc2-2746611adf5c admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Jul 24 00:54:04.248424 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 20/38] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:04.766144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-d8bfeccc-f48c-4a28-95bf-a04fbbf022c3 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:04.771720 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.wsgi [None req-d8bfeccc-f48c-4a28-95bf-a04fbbf022c3 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-d8bfeccc-f48c-4a28-95bf-a04fbbf022c3 admin admin] Request method failure captured: Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Accept: application/json Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Accept-Encoding: gzip, deflate Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Connection: keep-alive Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Content-Length: 0 Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Host: 10.222.0.18 Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Openstack-Api-Version: compute 2.90 Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Openstack-System-Scope: None Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: User-Agent: python-novaclient Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Auth-Token: gAAAAABi3Jgsvj-DTnIlYANm9e1AY-47kePuNjyHhjzFvIgwSUmr2-cFBsAQT9QkLuYw8H52tAeTIDhafhrzKcY54K9rHvgh_9CiCBjp9E6jzuDm8qnXot1ssPLtbZU0NgNyWa-GqteLb0qgJe9X2XWWIykkl_S90GUBL_JTAUB6iF8bsOAtlzw Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Domain-Id: None Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Domain-Name: None Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Identity-Status: Confirmed Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Is-Admin-Project: True Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Openstack-Nova-Api-Version: 2.90 Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Domain-Id: default Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Domain-Name: Default Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Name: admin Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Role: reader,member,admin Jul 24 00:54:04.775647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Roles: reader,member,admin Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_17ae0b4181524f419070ad7610eaf309", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant: admin Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant-Name: admin Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User: admin Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Domain-Id: default Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Domain-Name: Default Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Id: f00ceffcf1cf4b95908b3794e7b9b31d Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Name: admin Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: method: > Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: exception: Flavor 42 could not be found. Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: action_args: {'id': '42'} Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 00:54:04.777356 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:04.778413 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.779923 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-d8bfeccc-f48c-4a28-95bf-a04fbbf022c3 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103247) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 00:54:04.781239 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-d8bfeccc-f48c-4a28-95bf-a04fbbf022c3 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.391133 Jul 24 00:54:04.781239 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 19/39] 10.222.0.18 () {60 vars in 1190 bytes} [Sun Jul 24 00:54:04 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 392 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 24 00:54:04.786634 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-6615c17b-796c-4858-ad28-856ee23482bd admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:04.790484 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:54:04.810473 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-6615c17b-796c-4858-ad28-856ee23482bd admin admin] 10.222.0.18 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027350 Jul 24 00:54:04.811051 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 21/40] 10.222.0.18 () {60 vars in 1210 bytes} [Sun Jul 24 00:54:04 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:54:04.820922 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-6fed7801-4cd4-4d1d-bb57-ee97639f6092 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:04.841729 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-6fed7801-4cd4-4d1d-bb57-ee97639f6092 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024537 Jul 24 00:54:04.842252 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 20/41] 10.222.0.18 () {60 vars in 1210 bytes} [Sun Jul 24 00:54:04 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:54:04.852763 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-69a69ae1-7a95-4357-a060-4677be6849ff admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:04.858916 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.wsgi [None req-69a69ae1-7a95-4357-a060-4677be6849ff admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-69a69ae1-7a95-4357-a060-4677be6849ff admin admin] Request method failure captured: Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Accept: application/json Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Accept-Encoding: gzip, deflate Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Connection: keep-alive Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Content-Length: 0 Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Host: 10.222.0.18 Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Openstack-Api-Version: compute 2.90 Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Openstack-System-Scope: None Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: User-Agent: python-novaclient Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Auth-Token: gAAAAABi3Jgsvj-DTnIlYANm9e1AY-47kePuNjyHhjzFvIgwSUmr2-cFBsAQT9QkLuYw8H52tAeTIDhafhrzKcY54K9rHvgh_9CiCBjp9E6jzuDm8qnXot1ssPLtbZU0NgNyWa-GqteLb0qgJe9X2XWWIykkl_S90GUBL_JTAUB6iF8bsOAtlzw Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Domain-Id: None Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Domain-Name: None Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Identity-Status: Confirmed Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Is-Admin-Project: True Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Openstack-Nova-Api-Version: 2.90 Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Domain-Id: default Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Domain-Name: Default Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Name: admin Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Role: reader,member,admin Jul 24 00:54:04.862153 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Roles: reader,member,admin Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_17ae0b4181524f419070ad7610eaf309", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant: admin Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant-Name: admin Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User: admin Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Domain-Id: default Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Domain-Name: Default Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Id: f00ceffcf1cf4b95908b3794e7b9b31d Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Name: admin Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: method: > Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: exception: Flavor 42 could not be found. Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: action_args: {'id': '42'} Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 00:54:04.863613 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:04.864591 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:04.865951 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-69a69ae1-7a95-4357-a060-4677be6849ff admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103248) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 00:54:04.867786 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-69a69ae1-7a95-4357-a060-4677be6849ff admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015701 Jul 24 00:54:04.867786 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 22/42] 10.222.0.18 () {60 vars in 1190 bytes} [Sun Jul 24 00:54:04 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 24 00:54:05.820717 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-6fed7801-4cd4-4d1d-bb57-ee97639f6092 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:05.821723 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-6fed7801-4cd4-4d1d-bb57-ee97639f6092 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Jul 24 00:54:05.822117 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 21/43] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:06.347992 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-16c2ff28-4d4e-4411-9acc-720db0a23e5b admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:06.353587 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.wsgi [None req-16c2ff28-4d4e-4411-9acc-720db0a23e5b admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-16c2ff28-4d4e-4411-9acc-720db0a23e5b admin admin] Request method failure captured: Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Accept: application/json Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Accept-Encoding: gzip, deflate Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Connection: keep-alive Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Content-Length: 0 Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Host: 10.222.0.18 Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Openstack-Api-Version: compute 2.90 Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Openstack-System-Scope: None Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: User-Agent: python-novaclient Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Auth-Token: gAAAAABi3JgtH62NlY1w7hkSfBl7hJ4WHm9mt5acb-pEEgD9W3oqRjWIAZVe3aEyKHjWvLF2Edc8u3_nR7S7Ct5qNhJfJn3_IhyzJgDvqXsBvWLa0WuiGzkQDiwlVyPHNHh77yfz4ZstbUs5PsKaD-4MJSvqEOy8Uwh53-BABsyNoONHGfhTzgU Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Domain-Id: None Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Domain-Name: None Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Identity-Status: Confirmed Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Is-Admin-Project: True Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Openstack-Nova-Api-Version: 2.90 Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Domain-Id: default Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Domain-Name: Default Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Name: admin Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Role: reader,member,admin Jul 24 00:54:06.355149 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Roles: reader,member,admin Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_17ae0b4181524f419070ad7610eaf309", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant: admin Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant-Name: admin Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User: admin Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Domain-Id: default Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Domain-Name: Default Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Id: f00ceffcf1cf4b95908b3794e7b9b31d Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Name: admin Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: method: > Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: exception: Flavor 84 could not be found. Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: action_args: {'id': '84'} Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 00:54:06.356660 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:06.357491 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.358760 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-16c2ff28-4d4e-4411-9acc-720db0a23e5b admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103248) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 00:54:06.359987 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-16c2ff28-4d4e-4411-9acc-720db0a23e5b admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.398152 Jul 24 00:54:06.359987 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 23/44] 10.222.0.18 () {60 vars in 1190 bytes} [Sun Jul 24 00:54:05 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 399 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 24 00:54:06.366142 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-ecf4e955-0816-4dad-9bc8-0ad033d1b5c7 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:06.380567 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-ecf4e955-0816-4dad-9bc8-0ad033d1b5c7 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017977 Jul 24 00:54:06.381101 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 22/45] 10.222.0.18 () {60 vars in 1210 bytes} [Sun Jul 24 00:54:06 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:54:06.391314 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-cb40eee4-8edc-422b-b522-3e1913c3e5c2 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:06.405214 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-cb40eee4-8edc-422b-b522-3e1913c3e5c2 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017501 Jul 24 00:54:06.405747 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1210 bytes} [Sun Jul 24 00:54:06 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 00:54:06.415326 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-8b3f1701-2e72-4850-a43a-38b4c21e2214 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:06.420647 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.wsgi [None req-8b3f1701-2e72-4850-a43a-38b4c21e2214 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-8b3f1701-2e72-4850-a43a-38b4c21e2214 admin admin] Request method failure captured: Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Accept: application/json Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Accept-Encoding: gzip, deflate Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Connection: keep-alive Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Content-Length: 0 Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Host: 10.222.0.18 Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Openstack-Api-Version: compute 2.90 Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Openstack-System-Scope: None Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: User-Agent: python-novaclient Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Auth-Token: gAAAAABi3JgtH62NlY1w7hkSfBl7hJ4WHm9mt5acb-pEEgD9W3oqRjWIAZVe3aEyKHjWvLF2Edc8u3_nR7S7Ct5qNhJfJn3_IhyzJgDvqXsBvWLa0WuiGzkQDiwlVyPHNHh77yfz4ZstbUs5PsKaD-4MJSvqEOy8Uwh53-BABsyNoONHGfhTzgU Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Domain-Id: None Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Domain-Name: None Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Identity-Status: Confirmed Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Is-Admin-Project: True Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Openstack-Nova-Api-Version: 2.90 Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Domain-Id: default Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Domain-Name: Default Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Name: admin Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Role: reader,member,admin Jul 24 00:54:06.422144 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Roles: reader,member,admin Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_17ae0b4181524f419070ad7610eaf309", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant: admin Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant-Name: admin Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User: admin Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Domain-Id: default Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Domain-Name: Default Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Id: f00ceffcf1cf4b95908b3794e7b9b31d Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Name: admin Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: method: > Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: exception: Flavor 84 could not be found. Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: action_args: {'id': '84'} Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 00:54:06.423681 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:06.424684 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:06.425892 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-8b3f1701-2e72-4850-a43a-38b4c21e2214 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103247) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 00:54:06.426914 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-8b3f1701-2e72-4850-a43a-38b4c21e2214 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012084 Jul 24 00:54:06.426914 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 23/47] 10.222.0.18 () {60 vars in 1190 bytes} [Sun Jul 24 00:54:06 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 24 00:54:07.391056 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-cb40eee4-8edc-422b-b522-3e1913c3e5c2 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:07.392068 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-cb40eee4-8edc-422b-b522-3e1913c3e5c2 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001503 Jul 24 00:54:07.392506 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 25/48] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:07.912933 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-efb865f8-c976-459d-bf9b-c8a4441d96f8 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:07.919818 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.wsgi [None req-efb865f8-c976-459d-bf9b-c8a4441d96f8 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-efb865f8-c976-459d-bf9b-c8a4441d96f8 admin admin] Request method failure captured: Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Accept: application/json Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Accept-Encoding: gzip, deflate Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Connection: keep-alive Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Content-Length: 0 Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Host: 10.222.0.18 Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Openstack-Api-Version: compute 2.90 Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Openstack-System-Scope: None Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: User-Agent: python-novaclient Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Auth-Token: gAAAAABi3JgvJbVRjZb6l-yAQp1KMYTJb6XNV2U9EE0G9Im-Ce00b3m-q1bU7O2d92TGk_BbZsZgT2dEwqJpITURz3n8ZTiUnWnlp_gAzfLePVL1Hb9Bm9z4W8UGI_oK0dIyCoMTiv__r_STA4NVkFj8a0zawg8woN3LTCfvDE4k-PSJv62TGmo Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Domain-Id: None Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Domain-Name: None Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Identity-Status: Confirmed Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Is-Admin-Project: True Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Openstack-Nova-Api-Version: 2.90 Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Domain-Id: default Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Domain-Name: Default Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Project-Name: admin Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Role: reader,member,admin Jul 24 00:54:07.921310 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Roles: reader,member,admin Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_17ae0b4181524f419070ad7610eaf309", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant: admin Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-Tenant-Name: admin Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User: admin Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Domain-Id: default Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Domain-Name: Default Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Id: f00ceffcf1cf4b95908b3794e7b9b31d Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: X-User-Name: admin Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: method: > Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: exception: Flavor 451 could not be found. Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: action_args: {'id': '451'} Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 00:54:07.922679 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:07.923505 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.924855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-efb865f8-c976-459d-bf9b-c8a4441d96f8 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103247) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 00:54:07.926157 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-efb865f8-c976-459d-bf9b-c8a4441d96f8 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.390396 Jul 24 00:54:07.926157 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 24/49] 10.222.0.18 () {60 vars in 1193 bytes} [Sun Jul 24 00:54:07 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 391 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 24 00:54:07.932537 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-1317d9a8-c75f-40d5-bbd6-c9f6f7251770 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:07.946383 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-1317d9a8-c75f-40d5-bbd6-c9f6f7251770 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017600 Jul 24 00:54:07.946922 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 26/50] 10.222.0.18 () {60 vars in 1210 bytes} [Sun Jul 24 00:54:07 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:54:07.956344 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:07.971532 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018747 Jul 24 00:54:07.972066 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 25/51] 10.222.0.18 () {60 vars in 1210 bytes} [Sun Jul 24 00:54:07 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:54:07.981756 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-59c60e72-e838-4f9b-b514-8e8393477297 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:07.987054 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.wsgi [None req-59c60e72-e838-4f9b-b514-8e8393477297 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-59c60e72-e838-4f9b-b514-8e8393477297 admin admin] Request method failure captured: Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Accept: application/json Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Accept-Encoding: gzip, deflate Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Connection: keep-alive Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Content-Length: 0 Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Host: 10.222.0.18 Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Openstack-Api-Version: compute 2.90 Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Openstack-System-Scope: None Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: User-Agent: python-novaclient Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Auth-Token: gAAAAABi3JgvJbVRjZb6l-yAQp1KMYTJb6XNV2U9EE0G9Im-Ce00b3m-q1bU7O2d92TGk_BbZsZgT2dEwqJpITURz3n8ZTiUnWnlp_gAzfLePVL1Hb9Bm9z4W8UGI_oK0dIyCoMTiv__r_STA4NVkFj8a0zawg8woN3LTCfvDE4k-PSJv62TGmo Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Domain-Id: None Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Domain-Name: None Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Identity-Status: Confirmed Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Is-Admin-Project: True Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Openstack-Nova-Api-Version: 2.90 Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Domain-Id: default Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Domain-Name: Default Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Project-Name: admin Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Role: reader,member,admin Jul 24 00:54:07.988534 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Roles: reader,member,admin Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/17ae0b4181524f419070ad7610eaf309"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_17ae0b4181524f419070ad7610eaf309", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant: admin Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant-Id: 17ae0b4181524f419070ad7610eaf309 Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-Tenant-Name: admin Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User: admin Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Domain-Id: default Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Domain-Name: Default Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Id: f00ceffcf1cf4b95908b3794e7b9b31d Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: X-User-Name: admin Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: method: > Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: exception: Flavor 451 could not be found. Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: action_args: {'id': '451'} Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 00:54:07.990111 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:07.991205 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: ERROR nova.api.openstack.wsgi Jul 24 00:54:07.992726 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-59c60e72-e838-4f9b-b514-8e8393477297 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103248) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 00:54:07.994007 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-59c60e72-e838-4f9b-b514-8e8393477297 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.011596 Jul 24 00:54:07.994007 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 27/52] 10.222.0.18 () {60 vars in 1193 bytes} [Sun Jul 24 00:54:07 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 24 00:54:08.959319 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:08.960368 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001490 Jul 24 00:54:08.960762 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 26/53] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:09.481323 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-14164504-5519-4564-b345-89b5bef1dc23 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:54:09.498949 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-14164504-5519-4564-b345-89b5bef1dc23 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.397716 Jul 24 00:54:09.499478 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 28/54] 10.222.0.18 () {64 vars in 1235 bytes} [Sun Jul 24 00:54:09 2022] POST /compute/v2.1/flavors => generated 404 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:54:10.463732 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:10.464773 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001440 Jul 24 00:54:10.465205 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 27/55] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:10.984317 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-5c212704-a439-4b90-b67f-e83d14f55c2a admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:54:11.001458 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-5c212704-a439-4b90-b67f-e83d14f55c2a admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.399512 Jul 24 00:54:11.001997 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 29/56] 10.222.0.18 () {64 vars in 1235 bytes} [Sun Jul 24 00:54:10 2022] POST /compute/v2.1/flavors => generated 405 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:54:11.967655 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:11.968660 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Jul 24 00:54:11.969056 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 28/57] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:12.489322 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-2ec5a53c-25ec-47ab-b9e0-7809fb82e0bb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:54:12.506604 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-2ec5a53c-25ec-47ab-b9e0-7809fb82e0bb admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.399670 Jul 24 00:54:12.507173 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 30/58] 10.222.0.18 () {64 vars in 1235 bytes} [Sun Jul 24 00:54:12 2022] POST /compute/v2.1/flavors => generated 407 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:54:13.472051 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:13.473069 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Jul 24 00:54:13.473469 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 29/59] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 00:54:13.999743 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-0f0a0566-8f1d-404b-9b79-3f8806bce406 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 00:54:14.018593 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-0f0a0566-8f1d-404b-9b79-3f8806bce406 admin admin] 10.222.0.18 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.399831 Jul 24 00:54:14.019179 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 31/60] 10.222.0.18 () {64 vars in 1235 bytes} [Sun Jul 24 00:54:13 2022] POST /compute/v2.1/flavors => generated 409 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:54:15.005831 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:15.006873 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001432 Jul 24 00:54:15.007273 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 30/61] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:54:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:54:15.526610 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-cca7dd8d-b7fd-4159-946a-d741a0615ec1 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:54:15.545986 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-cca7dd8d-b7fd-4159-946a-d741a0615ec1 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.401768 Jul 24 00:54:15.546510 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 32/62] 10.222.0.18 () {60 vars in 1202 bytes} [Sun Jul 24 00:54:15 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:55:07.276678 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:55:07.277723 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-90276e72-58e1-4d62-9fb4-25496326f23a admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002002 Jul 24 00:55:07.278123 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 31/63] 10.222.0.18 () {54 vars in 930 bytes} [Sun Jul 24 00:55:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:55:07.288428 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:55:07.664592 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-af6c5f95-9276-425c-ba28-4061464073cc admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:55:07.684266 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-af6c5f95-9276-425c-ba28-4061464073cc admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.400200 Jul 24 00:55:07.684734 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 33/64] 10.222.0.18 () {60 vars in 1291 bytes} [Sun Jul 24 00:55:07 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:55:07.708533 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-b2016cb5-d44d-4e06-b72e-7e8c98b8ec4b admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:55:07.711976 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG dbcounter [-] [103247] Writing DB stats nova_api:SELECT=19,nova_api:SAVEPOINT=1,nova_api:INSERT=1,nova_api:RELEASE=1 {{(pid=103247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:55:07.718203 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-b2016cb5-d44d-4e06-b72e-7e8c98b8ec4b admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013264 Jul 24 00:55:07.718747 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 32/65] 10.222.0.18 () {60 vars in 1282 bytes} [Sun Jul 24 00:55:07 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:55:07.730429 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-9f9823e6-1736-4a08-a277-ba010d2f3a05 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:55:07.738006 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-9f9823e6-1736-4a08-a277-ba010d2f3a05 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011235 Jul 24 00:55:07.738519 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 34/66] 10.222.0.18 () {60 vars in 1285 bytes} [Sun Jul 24 00:55:07 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:55:07.750097 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:55:07.757989 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011687 Jul 24 00:55:07.758497 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 33/67] 10.222.0.18 () {60 vars in 1285 bytes} [Sun Jul 24 00:55:07 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 00:55:07.770160 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-c5edbd95-c21f-4aac-b176-18d1949a1a5b admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:55:07.778268 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-c5edbd95-c21f-4aac-b176-18d1949a1a5b admin admin] 10.222.0.18 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011576 Jul 24 00:55:07.778791 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 35/68] 10.222.0.18 () {60 vars in 1282 bytes} [Sun Jul 24 00:55:07 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 00:58:34.113784 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:34.114855 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Jul 24 00:58:34.115250 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 34/69] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:58:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:58:34.256563 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_api:SELECT=6 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:58:34.415754 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-4325870a-58f9-454a-99ae-2934ec78814f admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:34.416941 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:58:34.416941 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: warnings.warn(msg) Jul 24 00:58:34.418176 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-4325870a-58f9-454a-99ae-2934ec78814f admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:34.418619 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-4325870a-58f9-454a-99ae-2934ec78814f admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:34.419476 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-4325870a-58f9-454a-99ae-2934ec78814f admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:34.419908 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-4325870a-58f9-454a-99ae-2934ec78814f admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:34.454411 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-4325870a-58f9-454a-99ae-2934ec78814f admin admin] 10.222.0.18 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.199419 Jul 24 00:58:34.455070 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 36/70] 10.222.0.18 () {60 vars in 1193 bytes} [Sun Jul 24 00:58:34 2022] GET /compute/v2.1/os-services => generated 791 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 24 00:58:35.277738 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:35.278761 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001433 Jul 24 00:58:35.279194 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 35/71] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:58:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:58:35.472157 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-bc8334f4-db6c-4d80-b456-4a6c5a8465a9 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:35.473323 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:58:35.473323 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: warnings.warn(msg) Jul 24 00:58:35.474502 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-bc8334f4-db6c-4d80-b456-4a6c5a8465a9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:35.474995 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-bc8334f4-db6c-4d80-b456-4a6c5a8465a9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:35.475775 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-bc8334f4-db6c-4d80-b456-4a6c5a8465a9 admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:35.476200 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-bc8334f4-db6c-4d80-b456-4a6c5a8465a9 admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:35.495584 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-bc8334f4-db6c-4d80-b456-4a6c5a8465a9 admin admin] 10.222.0.18 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.075949 Jul 24 00:58:35.496058 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 37/72] 10.222.0.18 () {60 vars in 1193 bytes} [Sun Jul 24 00:58:35 2022] GET /compute/v2.1/os-services => generated 791 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 00:58:46.752674 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:46.753698 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001430 Jul 24 00:58:46.754093 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 36/73] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:58:46 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:58:46.898425 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_cell0:SELECT=4 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:58:46.899157 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_cell1:SELECT=4 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:58:46.899887 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG dbcounter [-] [103248] Writing DB stats nova_api:SELECT=8 {{(pid=103248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 00:58:46.948474 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-0aefd039-69fc-47bc-a424-3078f3bc68b8 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:46.949641 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:58:46.949641 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: warnings.warn(msg) Jul 24 00:58:46.950881 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-0aefd039-69fc-47bc-a424-3078f3bc68b8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:46.951324 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-0aefd039-69fc-47bc-a424-3078f3bc68b8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:46.952109 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-0aefd039-69fc-47bc-a424-3078f3bc68b8 admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:46.952540 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-0aefd039-69fc-47bc-a424-3078f3bc68b8 admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:46.980360 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-0aefd039-69fc-47bc-a424-3078f3bc68b8 admin admin] 10.222.0.18 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.086224 Jul 24 00:58:46.981058 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 38/74] 10.222.0.18 () {60 vars in 1193 bytes} [Sun Jul 24 00:58:46 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:58:47.806071 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: DEBUG nova.api.openstack.wsgi [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] Calling method '>' {{(pid=103247) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:47.807118 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: INFO nova.api.openstack.requestlog [None req-b2c7dcc4-4a9b-4a8f-bb27-acfdd201b9e9 admin admin] 10.222.0.18 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001413 Jul 24 00:58:47.807516 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: [pid: 103247|app: 0|req: 37/75] 10.222.0.18 () {56 vars in 1074 bytes} [Sun Jul 24 00:58:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 00:58:47.996578 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG nova.api.openstack.wsgi [None req-5d719580-c09c-4b6a-b199-9de9c0e18217 admin admin] Calling method '>' {{(pid=103248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 00:58:47.997889 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 24 00:58:47.997889 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: warnings.warn(msg) Jul 24 00:58:47.999121 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-5d719580-c09c-4b6a-b199-9de9c0e18217 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:47.999566 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-5d719580-c09c-4b6a-b199-9de9c0e18217 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:48.000369 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-5d719580-c09c-4b6a-b199-9de9c0e18217 admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:58:48.001385 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: DEBUG oslo_concurrency.lockutils [None req-5d719580-c09c-4b6a-b199-9de9c0e18217 admin admin] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:58:48.024061 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: INFO nova.api.openstack.requestlog [None req-5d719580-c09c-4b6a-b199-9de9c0e18217 admin admin] 10.222.0.18 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.078128 Jul 24 00:58:48.024548 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: [pid: 103248|app: 0|req: 39/76] 10.222.0.18 () {60 vars in 1193 bytes} [Sun Jul 24 00:58:47 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 00:59:43.253040 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jul 24 00:59:43.253796 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: Sun Jul 24 00:59:43 2022 - graceful shutdown triggered... Jul 24 00:59:43.254217 dv-cinder-master-3-iscsi devstack@n-api.service[103247]: Gracefully killing worker 1 (pid: 103247)... Jul 24 00:59:43.255195 dv-cinder-master-3-iscsi devstack@n-api.service[103248]: Gracefully killing worker 2 (pid: 103248)... Jul 24 00:59:44.254786 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: worker 1 buried after 1 seconds Jul 24 00:59:44.255338 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: worker 2 buried after 1 seconds Jul 24 00:59:44.255338 dv-cinder-master-3-iscsi devstack@n-api.service[103246]: goodbye to uWSGI. Jul 24 00:59:44.256421 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jul 24 00:59:44.256967 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jul 24 00:59:44.259760 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 24 00:59:44.270961 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 24 00:59:44 2022] *** Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: nodename: dv-cinder-master-3-iscsi Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: machine: x86_64 Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: clock source: unix Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: pcre jit disabled Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: detected number of CPU cores: 4 Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: current working directory: / Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: detected binary path: /usr/bin/uwsgi-core Jul 24 00:59:44.273154 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: your processes number limit is 47680 Jul 24 00:59:44.273873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: your memory page size is 4096 bytes Jul 24 00:59:44.273873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: detected max file descriptor number: 2048 Jul 24 00:59:44.273873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: lock engine: pthread robust mutexes Jul 24 00:59:44.273873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: thunder lock: enabled Jul 24 00:59:44.273873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 24 00:59:44.273873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 24 00:59:44.319466 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: Python main interpreter initialized at 0x5569326cf9c0 Jul 24 00:59:44.319466 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: python threads support enabled Jul 24 00:59:44.319466 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: your server socket listen backlog is limited to 100 connections Jul 24 00:59:44.319466 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: your mercy for graceful operations on workers is 90 seconds Jul 24 00:59:44.319873 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: mapped 403077 bytes (393 KB) for 2 cores Jul 24 00:59:44.319943 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: *** Operational MODE: preforking *** Jul 24 00:59:44.320182 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: *** uWSGI is running in multiple interpreter mode *** Jul 24 00:59:44.320182 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: spawned uWSGI master process (pid: 117941) Jul 24 00:59:44.320389 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 24 00:59:44.320953 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: spawned uWSGI worker 1 (pid: 117955, cores: 1) Jul 24 00:59:44.321309 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: spawned uWSGI worker 2 (pid: 117956, cores: 1) Jul 24 00:59:44.321461 dv-cinder-master-3-iscsi devstack@n-api.service[117941]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 24 00:59:44.979637 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 24 00:59:44.983179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 24 00:59:46.171614 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:59:46.171614 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: from cryptography.utils import int_from_bytes Jul 24 00:59:46.172236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:59:46.172236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: from cryptography.utils import int_from_bytes Jul 24 00:59:46.203990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:59:46.203990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: from cryptography.utils import int_from_bytes Jul 24 00:59:46.204728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 24 00:59:46.204728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: from cryptography.utils import int_from_bytes Jul 24 00:59:48.479254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 24 00:59:48.479855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 24 00:59:48.480025 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 24 00:59:48.480389 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 24 00:59:48.480646 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 24 00:59:48.481035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.481363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.481683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.482000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.482332 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.482642 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.482991 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.483313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.483640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.483901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.484216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.484539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.484859 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.485164 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.485474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.485794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.486099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.486420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.486724 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.487256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.487702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.488011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.488321 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.488664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.488979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.489288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.489611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.489920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.490260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.491176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.491585 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.492115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.492431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.492750 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.493163 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.493473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.493782 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.494103 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.494426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.494734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.495085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.495404 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.495723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.496043 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.496343 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.496649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.496956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.497274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.497578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.497881 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.498131 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.498436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.498797 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.499107 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.499431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.499687 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.499936 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.503906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.505386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.18 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.506665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.18 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.506665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.506665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.506665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.506665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.507207 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.507207 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.507457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.507766 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.508075 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.508397 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.508739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.509044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.509296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.509597 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.509904 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.510222 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.510526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.510854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.511160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.511459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.511763 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.512065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.512369 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.512672 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.512975 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.513277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.513579 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.513881 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.514182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.514502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.514846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.515154 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.515462 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.515769 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.516069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.516386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.516689 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.516993 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.517311 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.517614 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.517921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.518237 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.518545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.518871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.519190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.519491 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.519794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.520115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.520417 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.520719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.521022 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.521326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.521628 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.521968 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.522297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.522667 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.522995 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.523316 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.523643 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.523954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.524260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.524565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.524885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.525226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.525566 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.525912 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.526230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.526568 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.526927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.527251 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.527566 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.527886 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.528214 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.528531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.528847 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.529160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.529482 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.529801 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.530139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.530458 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.530798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.531119 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.531435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.531763 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.532089 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.532441 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.532727 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.533078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.533425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.533740 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.534071 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.534400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.534717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.535061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.535378 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.535708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.536044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.536359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.536681 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.536997 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.537313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.538208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.538616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.539189 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.539611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.540010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.540401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.540784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.541177 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.541564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.541888 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.542288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.542521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.542856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.543169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.543485 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.543795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.544121 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.544456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.544769 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.545110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.545427 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.545736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.546057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.546369 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.546682 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.547027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.547342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.547651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.547979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.548296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.548608 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.548936 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.549273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.549580 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.549900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.550259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.550526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.550864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.551186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.551514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.552380 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.552380 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.552588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.552867 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.553749 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.554129 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.554427 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.554747 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.555078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.555391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.555697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.556005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.556310 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.556622 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.556928 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.557233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.557535 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.557839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.558154 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.558462 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.558800 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.559104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.559408 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.559712 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.560060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.560369 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.560713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.561052 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.561373 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.561700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.562013 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.562329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.562639 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.562986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.563300 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.563611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.563934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.564246 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.564570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.564882 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.565208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.565519 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.565840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.566146 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.566464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.566831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.567170 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.567488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.567814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.568127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.568814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.569146 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.569467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.569777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.570091 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.570411 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.570739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.571077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.571389 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.571716 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.572027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.572365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.572652 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.572968 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.573296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.573630 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.573945 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.574256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.574578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.574915 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.575229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.575542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.575849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.576155 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.576483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.576785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.577129 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.577408 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.578285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.578285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.578458 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.578619 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.578952 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.579326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.579592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.579900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.580228 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.580598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.580898 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.582159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.582159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.582159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.582159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.582477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.582669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.583032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.583349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.583658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.583980 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.584327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.584640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.584958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.585273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.585580 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.585905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.586220 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.586531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.586868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.587182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.587492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.587805 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.588132 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.588444 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.588762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.589072 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.589379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.589689 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.590008 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.590589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.590949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.591269 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.591587 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.591694 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 24 00:59:48.592135 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 24 00:59:48.592234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.592234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.592401 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 24 00:59:48.592533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.592722 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 24 00:59:48.592865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.593064 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 24 00:59:48.593204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.593342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.593512 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.593670 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.593822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.593991 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.594210 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.594311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.594415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.594627 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.594728 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.594961 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.595065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.595281 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.595385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.595600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.595701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.595928 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.596032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.596212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.596330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.596503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.596598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.596809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.596901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.597115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.597218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.597429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.597532 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.597732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.597837 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.598042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.598145 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.598344 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.598491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.598663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.598757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.599035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.599132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.599292 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.599561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.599734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.600384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.600487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.600841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.600946 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.601172 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.601350 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.601504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.601696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.601839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.602017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.602186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.602328 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.602498 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.602670 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.602838 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.603040 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.603190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.603334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.603468 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.603638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.603776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.604023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.604168 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.604335 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.604470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.604645 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.604738 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.604948 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.605052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.605259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.605353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.605570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.605665 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.605882 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.605987 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.606193 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.606287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.606501 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.606679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.606840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.606967 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.607127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.607348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.607513 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.607661 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.607818 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.607973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.608121 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.608274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.608608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.608919 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.609179 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.609509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.609834 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.610145 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.610464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.610794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.611123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.611435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.611565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.611814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.611953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.612077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.612280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.612411 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.612629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.612757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.612906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.613048 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.613265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.613529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.613675 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.613910 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.614051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.614194 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.614456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.614598 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.614762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.615033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.615181 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.615389 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.616158 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.616420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.616668 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.616861 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.617056 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.617210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.617407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.617581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.617728 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.617880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.618024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.618190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.618341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.618517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.618669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.618882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.619051 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.619205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.619357 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.619503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.619666 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.619959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.620113 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.620265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.620408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.620551 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.620704 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.620866 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.621027 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.621839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.621839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.622138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.622282 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.622446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.622591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.622743 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.622930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.622930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.623225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.18 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.623225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.18 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.623532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.623756 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.623907 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.623907 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.624198 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.624352 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.624514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.624659 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.624828 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.625079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.625240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.625395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.625546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.625689 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.625840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.625992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.626145 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.626331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.626492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.626645 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.626808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.626972 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.627124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.627275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.627418 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.627560 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.627703 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.627856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.628001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.628159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.628310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.628463 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.628613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.628775 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.628919 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.629070 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.629213 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.629363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.629533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.630324 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.630482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.630634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.630895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.631050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.631230 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.631384 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.631384 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.631671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.631909 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.632062 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.632216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.632361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.632511 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.632663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.632815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.632958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.633110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.633270 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.633421 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.633572 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.633731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.633731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.634097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.634244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.634396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.634539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.634682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.634861 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.635019 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.635278 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.635447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.637517 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.637769 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.637936 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.638095 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.638255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.638411 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.638565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.638711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.638881 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.639026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.639183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.639354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.639501 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.639652 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.639795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.639937 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.640081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.640348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.640510 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.640668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.640812 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.640955 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.641098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.641262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.641262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.641557 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.641710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.641877 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.642115 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.642276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.642428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.642591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.642914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.643067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.643229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.643381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.643533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.643698 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.643698 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.643997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.644227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.644386 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.644539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.644683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.644827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.644976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.645130 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.645130 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.645521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.645715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.645876 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.645876 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.646178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.646178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.646467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.646618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.646799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.646799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 24 00:59:48.647111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.647111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.647442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.647618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.647776 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.647932 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648220 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648363 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648506 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648844 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.648999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.649150 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.649150 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.649441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.649599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.649762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.649914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.650066 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.650066 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.650386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.650640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.650813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.650976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.651123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.651266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.651417 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.651596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.651873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.652040 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.652244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.652395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.652549 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.652701 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.652871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.653088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.653239 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.653391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.653545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.653697 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.653849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.654004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.654148 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.654148 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.654538 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.654685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.654900 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.655051 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.655195 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.655339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.655339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.655747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.655897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656041 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656645 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.656938 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657089 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657089 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657541 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.657977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.658142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.658330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.658481 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.658625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.658784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.658945 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.659100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.659251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.659396 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.659539 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.659690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.659843 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.660010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.660155 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.660298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.660456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.660630 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.660874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661033 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661178 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661793 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.661945 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.662099 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.662242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.662971 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.663477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.663687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.663850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.663999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.664144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.664288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.664532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.664686 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.664831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.664831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.665123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.665275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.665432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.665432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.665706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.665706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666429 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666573 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666723 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.666921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.667222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.667420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.667420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.667699 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.667862 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.668095 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.668254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.668400 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.668552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.668703 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.668857 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669002 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669170 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669755 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.669900 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.670057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.670210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.670354 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.670516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.670796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.670975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.671220 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.671514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.671668 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.671821 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.671965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.672123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.672267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.672418 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.672569 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.672713 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.672864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.673017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.673162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.673394 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.673556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.673708 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.673860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674013 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674166 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674319 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.674878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675181 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675333 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675632 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675793 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.675956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.676332 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.676485 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.676638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.676792 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.676944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.676944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.677234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.677380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.677542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.677739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.677890 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.677890 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.678846 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.678998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.679143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.679365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.679512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.679664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.679809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.679960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.680114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.680403 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.680597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.680794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.680943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.681098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.681257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.681401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.681655 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.681814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.681961 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.682105 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.682263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.682422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.682575 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.682720 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.682941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.683102 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.683254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.683415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.683631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.683783 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.683934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.684088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.684245 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.684407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.684553 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.684723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.684723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685693 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.685997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.686141 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.686141 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.686543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.686690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.686877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687171 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687314 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687314 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.687892 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688187 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688505 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.688937 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689393 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689818 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.689968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.690113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.690275 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.690460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.690609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.690753 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.690921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.691065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.691225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.691369 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.691512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.691684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.691929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692397 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692540 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692833 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.692992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.693136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.693287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.693803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.693963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694420 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.695526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.695686 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.695832 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.695985 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.696139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.696283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.696531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.696700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.696700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.696984 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.697137 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.697297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.697442 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.697442 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.697721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.697880 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.698025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.698175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.698321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.698471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.698633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.698871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699024 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699323 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699468 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.699928 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.700079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.700231 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.700421 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.700582 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.700737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.700880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701031 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701031 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701303 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701751 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.701911 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.702159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.702317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.702473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.702618 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.702796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.702960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.703112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.703275 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.703436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.703594 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.703745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.703982 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.704148 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.704309 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.704467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.704618 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.704780 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.704929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.705095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.705095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.705477 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.705629 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.705786 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.705944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.706096 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.706255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.706408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.706582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.706582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.707194 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.707356 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.707521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.707683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.707843 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.707994 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.708153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.708319 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.708478 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.708636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.708796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.708947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.709097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.709097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.710123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.710280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.710439 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.710600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.710752 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.710935 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.711135 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.711300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.711452 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.711611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.711770 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.711929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.712093 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.712355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.712522 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.712688 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.712847 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713320 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.713963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.714200 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.714365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.714531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.714683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.714884 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.715041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.715191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.715358 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.715597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.716060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.716217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.716380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.716545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.716708 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.716885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.717139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.717298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.717298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.717601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.717759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.717759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.718046 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.718204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.718375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.718526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.718676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.718917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719084 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719237 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719555 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.719873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.720219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.720383 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.720548 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.720698 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.720873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721038 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721038 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721325 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.721955 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.722105 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.722254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.722519 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.722687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.722877 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723341 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.723947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.724097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.724348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.724516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.724688 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.724874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.725587 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.725819 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.725976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.726135 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.726297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.726297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.726601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.726759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.726965 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.727277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.727441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.727614 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.727767 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.727917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728068 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728232 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728232 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728527 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728686 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728837 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.728987 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729459 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729769 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.729920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.730222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.730382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.730541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.730700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.730960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.731116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.731266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.731416 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.731576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.731734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.731968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.732122 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.732279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.732430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.732698 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.732867 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733332 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733632 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733792 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.733957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.734109 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.734266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.734519 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.734678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.734885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.735052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.735203 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.735353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.735510 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.735669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.735863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.736112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.736271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.736430 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.736588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.736588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.736895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.737049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.737206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.737470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.737630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.737781 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.737940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.738113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.738430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.738589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.738753 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.738942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.739095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.739246 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.739395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.739544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.739704 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.739704 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.740010 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.740365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.740567 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.740725 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.740895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.741060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.741221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.741373 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.741531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.741691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.741891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.746743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.748150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.748313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.748468 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.748624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.748786 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.748949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.749108 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.749108 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.749410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.749563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.749731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.749883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.750175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.750336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.750488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.750488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.750828 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.750997 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.751150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.751309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.751473 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.751624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.751791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752338 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752490 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752648 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752810 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.752970 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.753129 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.753288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.753288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.753607 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.753760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.754087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.754255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.754408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.754605 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.754866 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.755148 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.755320 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.755473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.755633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.755784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.755960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.756180 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.756432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.756634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.756929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117956) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.758578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.761088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.762253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.763470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.764799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.764799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.764799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.764799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.764799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.765642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.765642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.765968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.766291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.766604 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.767007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.767256 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.767591 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.767903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.768213 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.768527 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.768861 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.769358 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.769699 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.770145 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.770574 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.770885 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.771211 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.771528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.772503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.772998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.773446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.773904 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.774224 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.774698 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.774928 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.775239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.775570 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.775893 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.776217 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.776526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.776858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.777189 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.777510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.777841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.778289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.778623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.778971 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.779304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.779640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.779956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.780274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.780595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.781007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.781205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.781537 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.781849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.782165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.782480 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.782809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.783126 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.783447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.783915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.784268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.784566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.784926 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.785224 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.785634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.785864 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.786197 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.786516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.786847 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.787163 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.788036 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.788568 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.789133 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.789472 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.789817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.790166 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.790484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.790862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.793030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.794370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.794370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.794370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.794370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.795234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.795234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.795712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.795712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.796127 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.796278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.796528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.796838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.797147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.797455 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.797809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.798117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.798457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.798812 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.799202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.799459 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.803941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.805271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.806388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.806388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.806388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.806388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.807032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.807032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.807350 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.807587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.807912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.808257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.808585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.808918 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.809659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.809659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.809972 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.810196 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.810518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.810927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.811453 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.811560 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.811908 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.812297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.812689 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.813029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.813344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.813665 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.813981 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.814462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.814803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.815148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.815470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.815816 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.816163 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.816553 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.816961 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.817150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.817456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.818024 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.818625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.819215 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.819814 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.820233 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.820566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.820897 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.821238 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.821553 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.822413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.822413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.822638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.822888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.823157 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.823470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.823777 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.824103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.824899 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.825514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.825873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.826202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.826533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.826954 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.827202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.827563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.827975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.828288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.828597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.828919 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.829237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.829544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.829852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.830174 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.830487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.830825 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.831132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.831442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.831772 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.832080 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.832388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.832693 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.833006 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.833313 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.833619 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.833927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.835174 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.835946 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.836266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.836617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.836892 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.837217 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.837526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.837841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.838199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.838542 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.838895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.839175 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.839503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.839822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.840160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.840563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.840877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.841292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.841614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.842201 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.842542 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.842893 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.843959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.843959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.843959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.844367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.844499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.845062 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.845243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.845537 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.845853 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.846171 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.846486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.846828 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.847152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.847453 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.847766 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.848079 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.848393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.848752 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.849014 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.849331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.849730 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.850960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.851346 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.851802 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.852200 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.852456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.852786 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.853104 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.853418 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.853740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.854076 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.854406 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.854705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.855215 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.855530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.855853 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.857202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.857202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.857202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.857202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.857651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.857755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.858025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.858361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.858711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.859075 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.859416 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.859719 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.860025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.860637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.860969 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.861298 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.861614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.861926 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.862233 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.862549 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.862891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.863202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.863511 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.863820 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.864136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.864463 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.864755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.865064 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.865372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.865921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.866456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.866791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.867115 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.867426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.867737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.868050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.868359 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.868700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.869039 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.869361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.869692 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.870007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.870338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.870653 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.871010 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.871342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.871663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.871981 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.872317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.872634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.872951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.873268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.873584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.873897 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.874214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.874550 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.874957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.875272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.875604 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.875943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.876249 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.876570 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.876886 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.877219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.877534 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.877848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.878162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.878476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.878806 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.879123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.879440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.879750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.880073 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.880390 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.880728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.881065 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.881794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.882091 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.882428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.882797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.883127 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.883478 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.883815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.884134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.884448 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.884757 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 24 00:59:48.885001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117955) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 24 00:59:48.900860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO dbcounter [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Registered counter for database nova_api Jul 24 00:59:48.922410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_db.sqlalchemy.engines [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117956) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:48.929630 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writer thread running {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:48.988450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.context [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a1472727-7c57-4e7e-873f-6498708782b3(cell1) {{(pid=117956) load_cells /opt/stack/nova/nova/context.py:464}} Jul 24 00:59:48.992553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:59:48.994178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:59:48.995010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:59:48.997024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:59:48.999011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO dbcounter [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Registered counter for database nova_cell0 Jul 24 00:59:49.002988 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO dbcounter [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Registered counter for database nova_cell1 Jul 24 00:59:49.010196 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_db.sqlalchemy.engines [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117956) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.011986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_db.sqlalchemy.engines [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117956) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.017441 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writer thread running {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.020207 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writer thread running {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.039374 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO dbcounter [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Registered counter for database nova_api Jul 24 00:59:49.059279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_db.sqlalchemy.engines [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117955) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.065916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writer thread running {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.123679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.context [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),a1472727-7c57-4e7e-873f-6498708782b3(cell1) {{(pid=117955) load_cells /opt/stack/nova/nova/context.py:464}} Jul 24 00:59:49.127661 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:59:49.129279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:59:49.130089 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 00:59:49.133156 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 00:59:49.135154 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO dbcounter [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Registered counter for database nova_cell0 Jul 24 00:59:49.139190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO dbcounter [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Registered counter for database nova_cell1 Jul 24 00:59:49.147468 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_db.sqlalchemy.engines [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117955) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.149334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_db.sqlalchemy.engines [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117955) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.155413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writer thread running {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.158232 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writer thread running {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.165994 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO dbcounter [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Registered counter for database nova_cell0 Jul 24 00:59:49.172505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_db.sqlalchemy.engines [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117956) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.176734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writer thread running {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.309984 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO dbcounter [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Registered counter for database nova_cell0 Jul 24 00:59:49.319310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_db.sqlalchemy.engines [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117955) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 24 00:59:49.324216 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writer thread running {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 24 00:59:49.494076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: WARNING keystonemiddleware.auth_token [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:59:49.501470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: WARNING keystonemiddleware.auth_token [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:59:49.649057 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: WARNING keystonemiddleware.auth_token [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:59:49.656605 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: WARNING keystonemiddleware.auth_token [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:59:49.842849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: WARNING keystonemiddleware.auth_token [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:59:49.849528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: WARNING keystonemiddleware.auth_token [None req-b278f91d-bf6b-429a-a6b5-8950b64b51a2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:59:49.856172 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5569326cf9c0 pid: 117956 (default app) Jul 24 00:59:50.003790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: WARNING keystonemiddleware.auth_token [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 24 00:59:50.010701 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: WARNING keystonemiddleware.auth_token [None req-ec43a781-6dde-4d2b-a260-b1342a9c2942 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 24 00:59:50.017010 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5569326cf9c0 pid: 117955 (default app) Jul 24 01:00:30.624243 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=3 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:30.627233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=3 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:30.629252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=3 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:30.631248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=3 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:32.776674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-48d9739c-6ad0-4e91-acd5-d70ca2754667 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Action: 'create', calling method: .version_select at 0x7f5e3252cd30>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1959204226"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:34.879360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-48d9739c-6ad0-4e91-acd5-d70ca2754667 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 4.269348 Jul 24 01:00:34.882315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 1/1] 10.222.0.18 () {60 vars in 1154 bytes} [Sun Jul 24 01:00:30 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 4274 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 24 01:00:35.151602 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ce49920c-ecb6-492d-9636-80406a5487e7 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e325f1820>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:35.182274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:35.183608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:35.188000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:35.189241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:35.540271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ce49920c-ecb6-492d-9636-80406a5487e7 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 200 len: 345 microversion: 2.1 time: 0.648422 Jul 24 01:00:35.541254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 1/2] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:00:34 2022] GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 345 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:36.555941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5bbf0c05-1c83-4dc3-b158-0177be5336a7 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e32388b80>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:36.781095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5bbf0c05-1c83-4dc3-b158-0177be5336a7 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 200 len: 346 microversion: 2.1 time: 0.231537 Jul 24 01:00:36.782067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:00:36 2022] GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 346 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:36.794706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-26bae4c8-76d4-4f30-9a7e-ddce80d469ab tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Action: 'create', calling method: .version_select at 0x7f5e323c8af0>, body: {"snapshot": {"volume_id": "aaf3c6f3-34ae-43d3-99cd-6131962b0aaf", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1591432046"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:37.772171 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-26bae4c8-76d4-4f30-9a7e-ddce80d469ab tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.981884 Jul 24 01:00:37.772762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 2/4] 10.222.0.18 () {60 vars in 1161 bytes} [Sun Jul 24 01:00:36 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:37.782482 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0f3d0148-a1d0-458c-a397-f23d0a713b1f tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e32388dc0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:37.988933 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0f3d0148-a1d0-458c-a397-f23d0a713b1f tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.209922 Jul 24 01:00:37.989677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 3/5] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:37 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:38.001177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a2ae2f8d-10a5-4a00-89eb-4cb3fa1abf9b tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3238e310>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:38.056444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a2ae2f8d-10a5-4a00-89eb-4cb3fa1abf9b tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.059216 Jul 24 01:00:38.057190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 3/6] 10.222.0.18 () {58 vars in 1139 bytes} [Sun Jul 24 01:00:37 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:38.068050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0f870025-35b0-40db-b4a0-751975760ff3 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3239d3a0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:38.123840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0f870025-35b0-40db-b4a0-751975760ff3 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.060161 Jul 24 01:00:38.125168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:38 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:39.135679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9a1d51f9-3331-4953-8cc9-ed2bfc049656 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3239d040>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:39.228264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9a1d51f9-3331-4953-8cc9-ed2bfc049656 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.089486 Jul 24 01:00:39.231618 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:39 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:40.245455 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d2307c57-b004-4f24-8c33-20ff7998df41 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3239da60>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:40.329596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d2307c57-b004-4f24-8c33-20ff7998df41 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.089128 Jul 24 01:00:40.331582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 5/9] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:40 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:41.343061 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d239151e-2bbc-4485-b282-0cd37298a137 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3238e9d0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:41.452133 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d239151e-2bbc-4485-b282-0cd37298a137 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 288 microversion: 2.1 time: 0.113856 Jul 24 01:00:41.452702 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 5/10] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:41 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 288 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:41.464995 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6319251d-0f2a-4e30-a3a3-9238907bb948 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3239dca0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:41.750555 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6319251d-0f2a-4e30-a3a3-9238907bb948 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "DELETE /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 202 len: 0 microversion: 2.1 time: 0.289206 Jul 24 01:00:41.751354 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 6/11] 10.222.0.18 () {58 vars in 1253 bytes} [Sun Jul 24 01:00:41 2022] DELETE /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 0 bytes in 291 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:00:41.760380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8490b438-07af-498e-a7ac-d496ab42d9d8 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3238e3a0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:41.813572 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8490b438-07af-498e-a7ac-d496ab42d9d8 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.056931 Jul 24 01:00:41.814729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:41 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:42.356837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-969046119", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "networks": [{"uuid": "b9d14848-c599-4212-8897-d0ad638b5278"}]}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:42.575870 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] validate_networks() for [('b9d14848-c599-4212-8897-d0ad638b5278', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:00:42.826521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-30fdb081-ce3f-4d6a-bdbe-d4ed907484a6 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e323c88b0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:42.876944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-30fdb081-ce3f-4d6a-bdbe-d4ed907484a6 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.054018 Jul 24 01:00:42.878018 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:42 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:43.889838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fecc7a14-3e38-4338-ab5c-5abcf85d1411 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e324ffca0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:43.980966 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fecc7a14-3e38-4338-ab5c-5abcf85d1411 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 200 len: 287 microversion: 2.1 time: 0.095037 Jul 24 01:00:43.981558 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 8/14] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:43 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 287 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:44.408297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-23908470-9366-424e-9f30-4664ef9bce46 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: .version_select at 0x7f5e3238e790>, body: {"keypair": {"name": "tempest-keypair-1776576010", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKR5Pr/oNvsbJ3SsI1ALErn69liKCfOX9/YTlrtlJOwGMtpHWghr4qDSxfUDxXrIKKb3IXndRAzHxd2FSsHUbR2ztOyE3zCLG3iHbP8MMc45jIrcG7O7jjUD36rjOtsuxA=="}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:44.543861 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-23908470-9366-424e-9f30-4664ef9bce46 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for project 644464d3831e468f8ab9957bb33817b0. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:00:44.558977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-23908470-9366-424e-9f30-4664ef9bce46 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for user a59f5944a0db4710a43a6447019864d2 and project 644464d3831e468f8ab9957bb33817b0. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:00:44.675538 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-23908470-9366-424e-9f30-4664ef9bce46 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.398103 Jul 24 01:00:44.676899 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 9/15] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:00:44 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:44.993529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-24bac4b7-68a5-4598-99ba-2fc007bdca4a tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3236bf70>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:45.066114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-24bac4b7-68a5-4598-99ba-2fc007bdca4a tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] HTTP exception thrown: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-24bac4b7-68a5-4598-99ba-2fc007bdca4a tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Request method failure captured: Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 HTTP/1.1 Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jmu4eVph7k8Z5IDQ-9elV5ypeLc616uPmQssHIlzrshAlVRQcFpG8QIePhU5zz9bvPzudI-8iDuxbasgQ3YoouYCvyuA65jbm3bKUxr_bVeI3Vco_IELq-gpvdYWKwHUK5FcN6y6OorCvbHEJtqgqwxwOV7bUmQBwbaluZRA8xxxgM Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:00:45.071957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_32d3a8e409644bc0a43293956ccbc83e", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: e6de928916e84bc28fa0204309f17575 Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0'} Jul 24 01:00:45.073241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. (HTTP 404) (Request-ID: req-7d8fea74-b315-4a9f-bc43-efcd8303913c) Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.073920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:00:45.074865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 24 01:00:45.075790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-24bac4b7-68a5-4598-99ba-2fc007bdca4a tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Returning 404 to user: Snapshot 28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:00:45.079731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-24bac4b7-68a5-4598-99ba-2fc007bdca4a tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0" status: 404 len: 111 microversion: 2.1 time: 0.088342 Jul 24 01:00:45.083571 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 10/16] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:00:44 2022] GET /compute/v2.1/os-snapshots/28b0a20b-b5ba-4b3e-b14c-22b63d3e3ff0 => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:00:45.099317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cf6b27e3-d4c7-4a22-bee9-281d0ab8b8ad tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e323c8b80>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:45.367146 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cf6b27e3-d4c7-4a22-bee9-281d0ab8b8ad tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "DELETE /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 202 len: 0 microversion: 2.1 time: 0.282186 Jul 24 01:00:45.371146 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 11/17] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:00:45 2022] DELETE /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 0 bytes in 287 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:00:45.407514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1fd552d3-e7e3-410b-8816-fff8d8b6cf4f tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3236b940>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:45.511832 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1fd552d3-e7e3-410b-8816-fff8d8b6cf4f tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 200 len: 345 microversion: 2.1 time: 0.139246 Jul 24 01:00:45.518263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 12/18] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:00:45 2022] GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 345 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:00:45.777010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:00:45.777576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:00:45.777918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:00:45.778333 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:00:45.778760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:00:45.825198 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:45.825656 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:45.826464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:45.843237 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:45.966101 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Getting quotas for project c129535452294d998fee4c9e259c9d41. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:00:46.010679 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Getting quotas for user b88e33e967ed47e780b3a1d64c5a0b8a and project c129535452294d998fee4c9e259c9d41. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:00:46.038606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:00:46.044401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:00:46.269041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a739fbea-1627-43e4-8870-7de866363a8d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.011852 Jul 24 01:00:46.270027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 7/19] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:00:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 4015 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:00:46.291513 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d791b368-ccd1-4013-bace-5e26b0c5b534 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:46.294367 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d791b368-ccd1-4013-bace-5e26b0c5b534 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:46.479687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-50234e05-a1f0-46cd-8eab-afb7546e4230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'create', calling method: .version_select at 0x7f5e31ae4b80>, body: {"keypair": {"name": "tempest-keypair-424227189", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGwJVuC5MYh8Wcx4Zy2jd+bCN9ntbvzL8v14f6vkHut+BfLDQXpb9f7f1ax007Nw2vzPqllgO1TmDoONLhvFeZqFRBNy56qMhIqmgobFQ5koPpYGxZMcmBEOSnSet+Bqlg=="}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:46.534304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-50234e05-a1f0-46cd-8eab-afb7546e4230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for project 75293bf0100f435bb567b690edc85533. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:00:46.549461 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-50234e05-a1f0-46cd-8eab-afb7546e4230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for user 9a9256b6d7e24c54b061673cd17217cd and project 75293bf0100f435bb567b690edc85533. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:00:46.603544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-50234e05-a1f0-46cd-8eab-afb7546e4230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.651210 Jul 24 01:00:46.604612 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 13/20] 10.222.0.18 () {62 vars in 1199 bytes} [Sun Jul 24 01:00:45 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 653 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 24 01:00:46.614421 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3e8a33ba-f03a-4ed9-8fb9-01817ad82bd4 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3236bd30>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:46.675669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-3e8a33ba-f03a-4ed9-8fb9-01817ad82bd4 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] HTTP exception thrown: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3e8a33ba-f03a-4ed9-8fb9-01817ad82bd4 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Request method failure captured: Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf HTTP/1.1 Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jmu4eVph7k8Z5IDQ-9elV5ypeLc616uPmQssHIlzrshAlVRQcFpG8QIePhU5zz9bvPzudI-8iDuxbasgQ3YoouYCvyuA65jbm3bKUxr_bVeI3Vco_IELq-gpvdYWKwHUK5FcN6y6OorCvbHEJtqgqwxwOV7bUmQBwbaluZRA8xxxgM Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:00:46.678177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_32d3a8e409644bc0a43293956ccbc83e", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: e6de928916e84bc28fa0204309f17575 Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'aaf3c6f3-34ae-43d3-99cd-6131962b0aaf'} Jul 24 01:00:46.679301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. (HTTP 404) (Request-ID: req-f6df87a0-3faf-41a7-ad53-f70c488287bd) Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.679948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:00:46.680903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 24 01:00:46.681799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.682668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.687689 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3e8a33ba-f03a-4ed9-8fb9-01817ad82bd4 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Returning 404 to user: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:00:46.688837 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3e8a33ba-f03a-4ed9-8fb9-01817ad82bd4 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 404 len: 109 microversion: 2.1 time: 0.083118 Jul 24 01:00:46.689944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 14/21] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:00:46 2022] GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 109 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:00:46.703895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c958ef34-9418-4004-9a12-a9bd99779a24 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e31a8e820>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:46.750289 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d791b368-ccd1-4013-bace-5e26b0c5b534 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:46.779832 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-c958ef34-9418-4004-9a12-a9bd99779a24 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] HTTP exception thrown: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c958ef34-9418-4004-9a12-a9bd99779a24 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Request method failure captured: Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf HTTP/1.1 Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jmu4eVph7k8Z5IDQ-9elV5ypeLc616uPmQssHIlzrshAlVRQcFpG8QIePhU5zz9bvPzudI-8iDuxbasgQ3YoouYCvyuA65jbm3bKUxr_bVeI3Vco_IELq-gpvdYWKwHUK5FcN6y6OorCvbHEJtqgqwxwOV7bUmQBwbaluZRA8xxxgM Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:00:46.784747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_32d3a8e409644bc0a43293956ccbc83e", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: e6de928916e84bc28fa0204309f17575 Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'aaf3c6f3-34ae-43d3-99cd-6131962b0aaf'} Jul 24 01:00:46.786753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. (HTTP 404) (Request-ID: req-aaa0bd44-7422-4028-ada5-e37ac72c367f) Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.802471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 24 01:00:46.803638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 24 01:00:46.806573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.808651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.809992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d791b368-ccd1-4013-bace-5e26b0c5b534 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1214 microversion: 2.1 time: 0.507657 Jul 24 01:00:46.810467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c958ef34-9418-4004-9a12-a9bd99779a24 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Returning 404 to user: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:00:46.810467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c958ef34-9418-4004-9a12-a9bd99779a24 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "DELETE /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 404 len: 109 microversion: 2.1 time: 0.090431 Jul 24 01:00:46.810467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 15/22] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:00:46 2022] DELETE /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 109 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:00:46.810938 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 8/23] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:46 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1214 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:00:46.811065 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b4dc160e-62cf-4eca-ab92-a599416b2f66 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Calling method '.version_select at 0x7f5e3239d160>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:46.844234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-b4dc160e-62cf-4eca-ab92-a599416b2f66 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] HTTP exception thrown: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b4dc160e-62cf-4eca-ab92-a599416b2f66 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Request method failure captured: Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf HTTP/1.1 Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jmu4eVph7k8Z5IDQ-9elV5ypeLc616uPmQssHIlzrshAlVRQcFpG8QIePhU5zz9bvPzudI-8iDuxbasgQ3YoouYCvyuA65jbm3bKUxr_bVeI3Vco_IELq-gpvdYWKwHUK5FcN6y6OorCvbHEJtqgqwxwOV7bUmQBwbaluZRA8xxxgM Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:00:46.846335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/32d3a8e409644bc0a43293956ccbc83e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_32d3a8e409644bc0a43293956ccbc83e", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 32d3a8e409644bc0a43293956ccbc83e Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-306132330 Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: e6de928916e84bc28fa0204309f17575 Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesSnapshotsTestJSON-306132330-project Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'aaf3c6f3-34ae-43d3-99cd-6131962b0aaf'} Jul 24 01:00:46.847384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. (HTTP 404) (Request-ID: req-4bc2fcd8-34f5-4852-abde-486a1c39cbfc) Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.848148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:00:46.849314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 24 01:00:46.850276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b4dc160e-62cf-4eca-ab92-a599416b2f66 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] Returning 404 to user: Volume aaf3c6f3-34ae-43d3-99cd-6131962b0aaf could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b4dc160e-62cf-4eca-ab92-a599416b2f66 tempest-VolumesSnapshotsTestJSON-306132330 tempest-VolumesSnapshotsTestJSON-306132330-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf" status: 404 len: 109 microversion: 2.1 time: 0.049092 Jul 24 01:00:46.851625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 16/24] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:00:46 2022] GET /compute/v2.1/os-volumes/aaf3c6f3-34ae-43d3-99cd-6131962b0aaf => generated 109 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:00:47.807139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2816794f-b068-4a0b-8766-5797c9acbdfa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:47.807738 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2816794f-b068-4a0b-8766-5797c9acbdfa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:48.058726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2816794f-b068-4a0b-8766-5797c9acbdfa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:48.072290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2816794f-b068-4a0b-8766-5797c9acbdfa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1214 microversion: 2.1 time: 0.269405 Jul 24 01:00:48.077826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 9/25] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:47 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1214 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:00:49.102179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:49.103016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:49.156893 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:49.165176 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:49.169307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:49.175337 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:50.263714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:50.308687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5ac69926-6484-4254-a678-701fb8aead13 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1210 microversion: 2.1 time: 1.219087 Jul 24 01:00:50.320776 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 17/26] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:49 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1210 bytes in 1232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:00:51.329233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:51.329962 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:51.352939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:51.356093 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:51.359629 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:51.362883 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:52.183853 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:52.238670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-18d6c7b2-0fec-492f-b559-2c7514e2e478 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1272 microversion: 2.1 time: 0.911524 Jul 24 01:00:52.249348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 10/27] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:51 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1272 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:00:53.257392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:53.258013 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:53.287377 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:53.287835 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:53.288626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:53.293642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:53.421781 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1969275405", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d2871582-ae5b-4292-87a4-e519f553496c"}], "security_groups": [{"name": "tempest-securitygroup--1044897734"}], "key_name": "tempest-keypair-1776576010", "user_data": "}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:54.037385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:54.050081 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-85b9e3bf-2f82-4ae0-9e0f-230109f9667c tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1272 microversion: 2.1 time: 0.797499 Jul 24 01:00:54.061075 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 18/28] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:53 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1272 bytes in 805 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:00:54.818316 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-405192399", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:55.067509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] validate_networks() for [('d2871582-ae5b-4292-87a4-e519f553496c', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:00:55.079883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] validate_networks() for [('fd9f01d9-09a8-410d-9e7d-efac033dcd4b', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:00:55.899219 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:56.569669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=8 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:00:57.867752 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:00:57.868883 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:00:57.871239 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:00:57.871885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:00:57.872532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:00:57.892162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:57.894280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:57.899294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:57.900996 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:57.948082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for project 644464d3831e468f8ab9957bb33817b0. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:00:57.957802 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for user a59f5944a0db4710a43a6447019864d2 and project 644464d3831e468f8ab9957bb33817b0. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:00:58.028556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:00:58.039363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:00:58.230956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-259367bc-21fa-4faf-ad22-cf69c46a63c0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 4.817127 Jul 24 01:00:58.231637 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 11/29] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:00:53 2022] POST /compute/v2.1/servers => generated 402 bytes in 4821 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 24 01:00:58.246572 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:58.247219 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:58.275815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:58.276259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:58.292017 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:58.292708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:59.101037 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:00:59.101652 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:00:59.102210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:00:59.102679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:00:59.103125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:00:59.141776 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:59.142640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:59.143761 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:00:59.145297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:00:59.214539 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:00:59.223616 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for user 283a1bfe0e3b45aaa0a33e5fe6b1d85a and project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:00:59.274651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:00:59.279056 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:00:59.439255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1635df6e-4e9d-4e2d-ba98-98c9d90e857a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 5.076968 Jul 24 01:00:59.440703 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 19/30] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:00:54 2022] POST /compute/v2.1/servers => generated 383 bytes in 5079 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 24 01:00:59.447227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1404991018", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e240c1c7-87a5-4124-b96b-a74d28ba2a48"}], "security_groups": [{"name": "tempest-securitygroup--602203386"}], "key_name": "tempest-keypair-424227189", "user_data": "}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:00:59.648633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:59.659196 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d9adf4a5-2e8e-4656-9a7f-37362c8a3070 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 1.423341 Jul 24 01:00:59.666197 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 12/31] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:58 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 1434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:00:59.683738 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2e520597-ded6-44c4-8b51-97efcab80c87 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:00:59.684358 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2e520597-ded6-44c4-8b51-97efcab80c87 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:00:59.759941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] validate_networks() for [('e240c1c7-87a5-4124-b96b-a74d28ba2a48', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:00:59.965073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2e520597-ded6-44c4-8b51-97efcab80c87 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:00:59.975044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2e520597-ded6-44c4-8b51-97efcab80c87 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1238 microversion: 2.1 time: 0.306770 Jul 24 01:00:59.976594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 13/32] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:59 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1238 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:00.007257 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-00be4577-f5be-47b4-8257-1c7a21acde95 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:00.007983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-00be4577-f5be-47b4-8257-1c7a21acde95 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:00.153683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-00be4577-f5be-47b4-8257-1c7a21acde95 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:00.172829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-00be4577-f5be-47b4-8257-1c7a21acde95 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1221 microversion: 2.1 time: 0.194889 Jul 24 01:01:00.174401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 14/33] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:00:59 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1221 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:00.741686 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-db1e6e15-9d74-448e-8f19-208b5473f81b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: .version_select at 0x7f5e31143ca0>, body: {"keypair": {"name": "tempest-keypair-2127023780", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPIFkKRASd1yasRb+9Z65dAnEjcTKB0Ro6RO3Yv69r5oDERNHMuHO2VwMMKAUANprUORvGClCfwT0TXvlcuItGTl2PJowQYM8KkfGtVBVvIgRXePC4OrQQVinJLuakqpGg=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:00.858094 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-db1e6e15-9d74-448e-8f19-208b5473f81b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:01:00.876997 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-db1e6e15-9d74-448e-8f19-208b5473f81b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:01:01.014213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-db1e6e15-9d74-448e-8f19-208b5473f81b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.838681 Jul 24 01:01:01.018168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 15/34] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:01:00 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:01.052017 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:01.055295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:01.081792 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:01.084366 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:01.100273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:01.103701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:03.214976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:03.243217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-bf77a102-fc5b-4914-917e-f83587b59489 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 2.214875 Jul 24 01:01:03.244904 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 16/35] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:01 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 2226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:03.256331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-687650307", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:03.367169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] validate_networks() for [('fd9f01d9-09a8-410d-9e7d-efac033dcd4b', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:01:04.495484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:01:04.496102 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:01:04.496326 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:01:04.496994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:01:04.497426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:01:04.523199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:04.525544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:04.530606 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:04.532392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:04.618512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for project 75293bf0100f435bb567b690edc85533. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:01:04.637544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for user 9a9256b6d7e24c54b061673cd17217cd and project 75293bf0100f435bb567b690edc85533. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:01:04.688990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:01:04.696269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:01:04.909052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fca80e42-5856-4394-b601-4125e1251959 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 5.466940 Jul 24 01:01:04.913927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 20/36] 10.222.0.18 () {62 vars in 1187 bytes} [Sun Jul 24 01:00:59 2022] POST /compute/v2.1/servers => generated 401 bytes in 5473 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 24 01:01:04.937899 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:04.938540 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:04.974444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:04.976853 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:04.980130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:04.981894 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:05.624250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:05.642501 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-064968a8-7ead-4879-8a0d-bc90735cf48a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1296 microversion: 2.1 time: 0.719761 Jul 24 01:01:05.643134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 21/37] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:04 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1296 bytes in 723 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:05.651608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:05.708875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:05.718367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=8 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:05.752678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:05.753137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:05.754712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:05.755231 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:07.924096 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:08.429022 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:01:08.429864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:01:08.434304 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:01:08.435184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:01:08.437865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:01:08.477632 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:08.480931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:08.487771 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:08.489086 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:08.585847 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:01:08.605634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for user 283a1bfe0e3b45aaa0a33e5fe6b1d85a and project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:01:08.652159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:01:08.656830 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:01:08.805534 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:08.813111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-27f54391-ff24-47c1-87cd-8f2b8c5bbddf tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 3.165975 Jul 24 01:01:08.822857 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-54ac9879-5d4e-45aa-8891-0051f9264d90 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 5.551652 Jul 24 01:01:08.822857 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 17/38] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:01:03 2022] POST /compute/v2.1/servers => generated 383 bytes in 5575 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:01:08.826142 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 22/39] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:05 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 3180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:08.835268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:08.837723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:08.854226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:08.854705 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:08.855550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:08.855998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:08.860164 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:08.869561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:08.881000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:08.881676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:08.883227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:08.884660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:09.070198 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:09.098317 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:09.108528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:09.115714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4a68a3bd-e02b-4319-b0e3-0c7259a78b3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1339 microversion: 2.20 time: 0.293053 Jul 24 01:01:09.132973 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 18/40] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:08 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1339 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:09.135957 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9c03a865-6e3d-4bc4-a6c1-bb7817855b8c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:09.136572 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9c03a865-6e3d-4bc4-a6c1-bb7817855b8c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:09.633330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9c03a865-6e3d-4bc4-a6c1-bb7817855b8c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:09.642633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9c03a865-6e3d-4bc4-a6c1-bb7817855b8c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1221 microversion: 2.1 time: 0.518225 Jul 24 01:01:09.643352 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 19/41] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:09 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1221 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:09.862151 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:09.871129 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ecbb4695-eb63-4327-bfcc-0ac3f630b80f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 1.044180 Jul 24 01:01:09.883751 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 23/42] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:08 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 1057 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:09.915053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:09.917923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:09.932705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:09.944682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:09.952104 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:09.955311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:09.981011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-ec0f362c-c77e-4468-9696-1a276b9b2cab req-9bdf837e-0f7a-41af-9c83-e2c5ff143e01 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dfcb8b81-6e4b-47a1-9b46-fa2088dbc613", "tag": "53ce7301-1644-4424-b492-4f95801956b5"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:10.005023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ec0f362c-c77e-4468-9696-1a276b9b2cab req-9bdf837e-0f7a-41af-9c83-e2c5ff143e01 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.005479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ec0f362c-c77e-4468-9696-1a276b9b2cab req-9bdf837e-0f7a-41af-9c83-e2c5ff143e01 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.042572 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-ec0f362c-c77e-4468-9696-1a276b9b2cab req-9bdf837e-0f7a-41af-9c83-e2c5ff143e01 service nova] Creating event network-changed:53ce7301-1644-4424-b492-4f95801956b5 for instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 on hv-cinder-maste Jul 24 01:01:10.043149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ec0f362c-c77e-4468-9696-1a276b9b2cab req-9bdf837e-0f7a-41af-9c83-e2c5ff143e01 service nova] Acquired lock "compute-rpcapi-router" {{(pid=117956) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 24 01:01:10.044312 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.052210 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.053211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.053689 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.074057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.compute.rpcapi [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 24 01:01:10.079666 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] Releasing lock "compute-rpcapi-router" {{(pid=117956) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 24 01:01:10.115750 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-941216ac-79d8-4e56-a678-f7d9e6febf92 None None] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.452091 Jul 24 01:01:10.116705 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 20/43] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:10.141405 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:10.142440 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:10.154081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.155363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.157769 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.158411 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.502605 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:10.512100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:10.514016 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:10.532311 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6ac01ff2-7b76-4c34-9fee-c986612a34bb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1339 microversion: 2.20 time: 0.395104 Jul 24 01:01:10.537871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 21/44] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:10 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1339 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:01:10.663860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:10.664841 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:10.679191 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.679624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.680408 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.680846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.948049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:10.955687 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e9429653-ae4e-4aa6-80ef-bf6da367821b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1291 microversion: 2.1 time: 0.296934 Jul 24 01:01:10.979176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 22/45] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:10 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1291 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:10.981931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:10.984098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:10.984758 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:10.991470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.991871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.992667 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:10.993120 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:10.995690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-96aa026b-4306-4984-b3f3-fde4207ddf71 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 1.093758 Jul 24 01:01:10.996433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 24/46] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:09 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 1112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:11.324838 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:11.341581 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6cda6f55-9f41-40ca-9c8b-96d4dd043f14 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.361168 Jul 24 01:01:11.352048 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 23/47] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:10 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:11.553172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:11.553814 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:11.567901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:11.569104 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:11.571349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:11.572129 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:11.751805 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:11.760970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:11.762827 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:11.764176 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-47d44a45-6368-4f27-badc-ada77f67de25 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.215666 Jul 24 01:01:11.764913 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 25/48] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:11 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:11.978428 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:11.978428 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:11.979675 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=10 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:11.983617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:11.984065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:11.984863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:11.985295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:11.991061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=53 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:12.013639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:12.014852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:12.020940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.021616 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.022682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.023159 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.069460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:12.077855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8cac7fef-d623-4800-9c7b-e237a79c2733 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1279 microversion: 2.1 time: 0.108042 Jul 24 01:01:12.078805 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 24/49] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:11 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1279 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:12.190169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:12.200111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cca5bcc5-83bc-4791-bdbd-954691206de8 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 0.190569 Jul 24 01:01:12.201101 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 26/50] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:12 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:12.306005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-b391e8c7-6c96-4bde-b4f7-65b4b1b3273a req-2cb618e7-0256-4029-92fd-bce197d62910 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "97d6538b-2585-40fc-b68c-e9a088724a39", "tag": "7813289d-8139-4d10-ab8e-926e1b2e6577"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:12.320257 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-b391e8c7-6c96-4bde-b4f7-65b4b1b3273a req-2cb618e7-0256-4029-92fd-bce197d62910 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.327503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-b391e8c7-6c96-4bde-b4f7-65b4b1b3273a req-2cb618e7-0256-4029-92fd-bce197d62910 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.345402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-b391e8c7-6c96-4bde-b4f7-65b4b1b3273a req-2cb618e7-0256-4029-92fd-bce197d62910 service nova] Creating event network-changed:7813289d-8139-4d10-ab8e-926e1b2e6577 for instance 97d6538b-2585-40fc-b68c-e9a088724a39 on hv-cinder-maste Jul 24 01:01:12.352641 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-b391e8c7-6c96-4bde-b4f7-65b4b1b3273a req-2cb618e7-0256-4029-92fd-bce197d62910 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055851 Jul 24 01:01:12.362336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:12.364132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:12.366850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 25/51] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:12.376703 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.377399 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.389660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.406270 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.735864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-5b44f8d5-02d6-4283-9d2a-c0abfffa75ec req-0fccdd5c-82f1-4ae6-af47-9c8c872c4425 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64ba3b57-16f8-4b6f-b0a6-d0141afc0faa", "tag": "710ccda1-d8a7-4b65-9124-7d15ed6b5d75"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:12.747420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-5b44f8d5-02d6-4283-9d2a-c0abfffa75ec req-0fccdd5c-82f1-4ae6-af47-9c8c872c4425 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.747877 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-5b44f8d5-02d6-4283-9d2a-c0abfffa75ec req-0fccdd5c-82f1-4ae6-af47-9c8c872c4425 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.770047 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-5b44f8d5-02d6-4283-9d2a-c0abfffa75ec req-0fccdd5c-82f1-4ae6-af47-9c8c872c4425 service nova] Creating event network-changed:710ccda1-d8a7-4b65-9124-7d15ed6b5d75 for instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa on hv-cinder-maste Jul 24 01:01:12.775963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-5b44f8d5-02d6-4283-9d2a-c0abfffa75ec req-0fccdd5c-82f1-4ae6-af47-9c8c872c4425 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.162792 Jul 24 01:01:12.776939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 26/52] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:01:12.785093 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:12.786197 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:12.793834 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.794576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:12.795550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:12.796132 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:13.277270 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:13.337456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-164f2871-0a5c-43c2-ae7f-544eecb1008b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.981125 Jul 24 01:01:13.339418 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 27/53] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:12 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:13.361794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:13.365597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:13.372258 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:13.373074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:13.374011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:13.374620 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:13.415163 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:13.428564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:13.431901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:13.434135 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b2ab4275-fe6f-4e54-99cd-cf4fde52e551 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.653251 Jul 24 01:01:13.435488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 27/54] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:12 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:13.442094 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:13.443152 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:13.451085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:13.452009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:13.452938 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:13.453561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:13.952449 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:13.962094 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-98e464e6-a14e-498e-b1ce-545599ea90b3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 0.525292 Jul 24 01:01:13.964204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 28/55] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:13 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:13.970556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-554821368", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "37cd1572-14d0-471e-b2ca-ff3f9078882e"}], "security_groups": [{"name": "tempest-securitygroup--2089527955"}], "key_name": "tempest-keypair-2127023780", "user_data": "}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:14.227032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:14.252470 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d4d43298-d847-402b-b1c9-df71a40992e4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.907930 Jul 24 01:01:14.264471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 28/56] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:13 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:14.355804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] validate_networks() for [('37cd1572-14d0-471e-b2ca-ff3f9078882e', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:01:14.357216 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:14.357842 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:14.365373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:14.365837 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:14.366630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:14.367100 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:14.569708 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=4 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:14.669713 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=12 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:14.740902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:14.753044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c11633fc-965f-4400-b95b-ea11b243a5b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.396340 Jul 24 01:01:14.754279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 29/57] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:14 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:14.775677 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:14.779561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:14.790649 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:14.791647 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:14.793168 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:14.793789 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.461784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:15.467344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=75 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:15.482593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:15.485947 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:15.488447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e9b767ad-4e94-4d81-b403-d3d6594164e1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.731247 Jul 24 01:01:15.492989 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 30/58] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:14 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:15.499880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-e0427341-8217-4647-aab2-33a2bd81294f req-f42d2561-9dd9-451b-a2df-84c437b73c59 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:15.527362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-e0427341-8217-4647-aab2-33a2bd81294f req-f42d2561-9dd9-451b-a2df-84c437b73c59 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.527803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-e0427341-8217-4647-aab2-33a2bd81294f req-f42d2561-9dd9-451b-a2df-84c437b73c59 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.551112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:01:15.553331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:01:15.554700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:01:15.555711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:01:15.556036 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:01:15.567289 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=33 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:15.572702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.574267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.575363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.576033 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.581696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-e0427341-8217-4647-aab2-33a2bd81294f req-f42d2561-9dd9-451b-a2df-84c437b73c59 service nova] Creating event network-changed:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:01:15.582237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-e0427341-8217-4647-aab2-33a2bd81294f req-f42d2561-9dd9-451b-a2df-84c437b73c59 service nova] Acquired lock "compute-rpcapi-router" {{(pid=117955) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 24 01:01:15.594673 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.594960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.596235 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.596809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.607949 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.compute.rpcapi [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 24 01:01:15.608830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] Releasing lock "compute-rpcapi-router" {{(pid=117955) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 24 01:01:15.616213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:01:15.621759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:01:15.632782 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d5f871bf-bb9c-4ba5-93a4-c0c8d8040d80 None None] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.137103 Jul 24 01:01:15.634004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 31/59] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:15.640785 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:15.641616 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:15.645926 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:01:15.648276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:01:15.651122 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.651768 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.652544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.653299 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.715864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-18aaa6c4-4b48-42bd-8048-8c3ff636e349 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.750011 Jul 24 01:01:15.716826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 29/60] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:01:13 2022] POST /compute/v2.1/servers => generated 402 bytes in 1752 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:01:15.721447 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:15.723774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:15.733080 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.733685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:15.735261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:15.735936 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:16.527503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:16.537531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-33e745a9-18da-4248-8062-ed3255a32b30 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.819634 Jul 24 01:01:16.541008 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 30/61] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:15 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 823 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:16.571763 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:16.572410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:16.585063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:16.585785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:16.586922 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:16.587615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:16.792074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:16.803248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-83498b6b-e858-4c34-a998-35c9a58ab2c3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1237 microversion: 2.1 time: 0.261049 Jul 24 01:01:16.805317 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 31/62] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:16 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1237 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:16.812602 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:16.822833 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0a927caa-2146-4823-8029-9d1560b871fe tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 1.187354 Jul 24 01:01:16.823318 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:16.827040 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:16.827685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 32/63] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:15 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 1193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:16.835784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:16.837724 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:16.838058 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:16.838885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:16.839259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:16.839375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:16.849507 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:16.850279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:16.852038 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:16.852729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.214711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:17.224582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-403a030d-b9f5-4e9c-8dc3-5c40d0bd92ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.415320 Jul 24 01:01:17.225867 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 32/64] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:16 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:17.348297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-dc6c5c2c-3268-459a-82e9-843cad05d32c req-bf49ac5c-cffd-4773-8415-0312e691579a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfcb8b81-6e4b-47a1-9b46-fa2088dbc613", "name": "network-vif-plugged", "status": "completed", "tag": "53ce7301-1644-4424-b492-4f95801956b5"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:17.357219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:17.366795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:17.371311 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-dc6c5c2c-3268-459a-82e9-843cad05d32c req-bf49ac5c-cffd-4773-8415-0312e691579a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.372579 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:17.373056 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-dc6c5c2c-3268-459a-82e9-843cad05d32c req-bf49ac5c-cffd-4773-8415-0312e691579a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.373716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4603468f-be19-43f6-8290-6a9dc09c2435 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.544698 Jul 24 01:01:17.391083 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 33/65] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:16 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:17.421063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-dc6c5c2c-3268-459a-82e9-843cad05d32c req-bf49ac5c-cffd-4773-8415-0312e691579a service nova] Creating event network-vif-plugged:53ce7301-1644-4424-b492-4f95801956b5 for instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 on hv-cinder-maste Jul 24 01:01:17.433347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-dc6c5c2c-3268-459a-82e9-843cad05d32c req-bf49ac5c-cffd-4773-8415-0312e691579a service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.205548 Jul 24 01:01:17.434563 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 33/66] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:17.556951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:17.557721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:17.569290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.569891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.571065 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.571613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.658333 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-afd02e56-6e5c-463b-876f-09d834598f1d req-43d29bd2-2484-4a27-bdc9-4d64a8c44f5c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4389fd86-dd98-40ca-8e28-a9153502931c", "tag": "dd19007e-705d-4543-9981-8c458087a8e3"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:17.670805 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-afd02e56-6e5c-463b-876f-09d834598f1d req-43d29bd2-2484-4a27-bdc9-4d64a8c44f5c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.671260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-afd02e56-6e5c-463b-876f-09d834598f1d req-43d29bd2-2484-4a27-bdc9-4d64a8c44f5c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.684726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-afd02e56-6e5c-463b-876f-09d834598f1d req-43d29bd2-2484-4a27-bdc9-4d64a8c44f5c service nova] Creating event network-changed:dd19007e-705d-4543-9981-8c458087a8e3 for instance 4389fd86-dd98-40ca-8e28-a9153502931c on hv-cinder-maste Jul 24 01:01:17.691547 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-afd02e56-6e5c-463b-876f-09d834598f1d req-43d29bd2-2484-4a27-bdc9-4d64a8c44f5c service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038810 Jul 24 01:01:17.692473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 34/67] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:17.775329 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:17.784896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-20275a28-2e63-4c70-843c-381096452cf9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.231621 Jul 24 01:01:17.786221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 34/68] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:17 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:17.823766 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:17.824870 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:17.831930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.832502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.833418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.833930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.845630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:17.847151 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:17.862098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.863280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.864234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:17.864770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:17.951759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:17.963066 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6e566cd5-0e39-4121-be60-416dffb44838 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1299 microversion: 2.1 time: 0.143853 Jul 24 01:01:17.963653 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 35/69] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:17 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1299 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:18.043247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:18.052466 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-25c0418a-d6ab-430c-a04c-21b470b460fa tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 0.211738 Jul 24 01:01:18.054807 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 35/70] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:17 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:18.242706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:18.243388 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:18.255744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.256211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.256976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.257424 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.404398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:18.405883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:18.421900 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.422545 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.423634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.424268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.736838 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:18.746521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-20e88b9d-4e23-49fa-a23d-d3063697678e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.508582 Jul 24 01:01:18.756838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:18.757402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 36/71] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:18 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:18.767251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:18.770519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:18.772349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7162d03a-9099-4f7f-89bd-49ea222c2745 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.382010 Jul 24 01:01:18.775534 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 36/72] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:18 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:18.803425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:18.804035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:18.810853 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.811307 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.812242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.812760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.981904 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:18.982517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:18.988574 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.989137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:18.989909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:18.990344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:19.003897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:19.013045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6bf84004-a946-44c7-95d1-01e942f2f850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.214222 Jul 24 01:01:19.014244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 37/73] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:18 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:19.070959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:19.071607 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:19.085713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:19.086165 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:19.087143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:19.087731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:19.110316 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:19.127834 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-eef79adb-bb37-4925-8f91-af21970c901a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1299 microversion: 2.1 time: 0.150116 Jul 24 01:01:19.129468 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 37/74] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:18 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1299 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:19.350134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:19.359112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fa652c0d-753f-46ac-b92d-b781f6f740b1 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 0.293172 Jul 24 01:01:19.381326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 38/75] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:19 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:19.764099 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:19.764915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:19.774331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:19.774805 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:19.783391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:19.784132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:19.792959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:19.793567 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:19.801431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:19.808164 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:19.809258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:19.809851 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.218794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.230354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-eb7cb2e9-fc34-4218-9a1a-51caf8cf4a0f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.469937 Jul 24 01:01:20.241587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 38/76] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:19 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:20.241587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:20.241587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:20.248115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.249093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:20.250311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.251371 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:20.252527 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.258905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.269815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.272077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3190a4aa-0691-4de8-a8da-29d76091a3b7 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.482966 Jul 24 01:01:20.276196 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 39/77] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:19 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:01:20.284137 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:20.286623 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:20.295343 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:20.302699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.303582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:20.304031 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.433374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.449453 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-14e2572d-7414-47fb-a2ae-d501eaf51fa6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1299 microversion: 2.1 time: 0.168934 Jul 24 01:01:20.451151 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 40/78] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:20 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1299 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:20.464989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:20.465606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:20.473619 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:20.474289 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.475362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:20.475949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:20.571923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.587424 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e757b50e-9717-4aa5-b737-ce6768eb2f55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.354314 Jul 24 01:01:20.596988 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 39/79] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:20 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:20.664534 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:20.673235 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7d292f36-8c4f-45bd-ad4b-82c385f1622d tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 0.220287 Jul 24 01:01:20.675259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 41/80] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:20 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:21.082084 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-71e1a25b-e86a-49ca-89be-d1e2ba4f16af req-6699d09f-a844-4796-a04d-72a55ebd3c27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64ba3b57-16f8-4b6f-b0a6-d0141afc0faa", "name": "network-vif-plugged", "status": "completed", "tag": "710ccda1-d8a7-4b65-9124-7d15ed6b5d75"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:21.104838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-71e1a25b-e86a-49ca-89be-d1e2ba4f16af req-6699d09f-a844-4796-a04d-72a55ebd3c27 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.105292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-71e1a25b-e86a-49ca-89be-d1e2ba4f16af req-6699d09f-a844-4796-a04d-72a55ebd3c27 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.132591 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-71e1a25b-e86a-49ca-89be-d1e2ba4f16af req-6699d09f-a844-4796-a04d-72a55ebd3c27 service nova] Creating event network-vif-plugged:710ccda1-d8a7-4b65-9124-7d15ed6b5d75 for instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa on hv-cinder-maste Jul 24 01:01:21.139642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-71e1a25b-e86a-49ca-89be-d1e2ba4f16af req-6699d09f-a844-4796-a04d-72a55ebd3c27 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066414 Jul 24 01:01:21.146376 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 40/81] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:21.250185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:21.255939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:21.283409 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.290986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.293755 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.294732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:21.295822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:21.304293 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.312958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.313412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.314199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.314633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.813047 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:21.822078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-40a7b9d7-4d3e-4996-9e39-9d20460e8238 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.576631 Jul 24 01:01:21.825477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 42/82] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:21 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:21.838101 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:21.840018 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:21.855347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.855762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.858276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.858809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.909030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:21.919047 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:21.920585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:21.921715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9eaf437b-7852-4766-9db1-2cfd0d61faf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.632041 Jul 24 01:01:21.922285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 41/83] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:21 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:21.952573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:21.953344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:21.966790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.967255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:21.968050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:21.968491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.264111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:22.275198 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d333a711-f8ae-4180-b68e-bc53d6100b1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.448553 Jul 24 01:01:22.275788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 43/84] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:21 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:22.283921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:22.284583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:22.287847 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:22.291901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.292841 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.294066 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.294671 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.296278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2d119867-8864-41d6-ae5d-5c6aa8128676 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.363787 Jul 24 01:01:22.298890 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=100 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:22.303849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 42/85] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:21 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:22.317123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-7f6bec94-2e3d-494c-9dc5-e27653485476 req-360f7db2-fe50-40b1-b393-d0df868f698c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c006a24-d0c5-4ab6-9ce4-259e05a9c56a", "tag": "3e10a0f3-72ea-499e-9aa7-f84921595c2e"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:22.330804 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-7f6bec94-2e3d-494c-9dc5-e27653485476 req-360f7db2-fe50-40b1-b393-d0df868f698c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.332247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-7f6bec94-2e3d-494c-9dc5-e27653485476 req-360f7db2-fe50-40b1-b393-d0df868f698c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.350384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-7f6bec94-2e3d-494c-9dc5-e27653485476 req-360f7db2-fe50-40b1-b393-d0df868f698c service nova] Creating event network-changed:3e10a0f3-72ea-499e-9aa7-f84921595c2e for instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a on hv-cinder-maste Jul 24 01:01:22.355483 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-7f6bec94-2e3d-494c-9dc5-e27653485476 req-360f7db2-fe50-40b1-b393-d0df868f698c service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050398 Jul 24 01:01:22.356680 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 43/86] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:22.543523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:22.553137 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c40d6b38-9656-41f0-a421-e0e2483a0583 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1314 microversion: 2.1 time: 0.276086 Jul 24 01:01:22.570097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 44/87] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:22 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1314 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:22.841017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:22.841944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:22.850643 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.851083 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.851856 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.858933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.941772 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:22.943639 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:22.950591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.951367 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:22.952377 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:22.953644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.363776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.372365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.372683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.374434 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.375754 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-48bf5f54-dcce-43cf-8dc9-e27790d86082 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.439574 Jul 24 01:01:23.379325 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 45/88] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:22 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:23.384184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-6f12fa15-058a-4b30-af83-1aab012af940 req-83669aaf-5292-4cf7-b67c-0c3c907c4afe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "97d6538b-2585-40fc-b68c-e9a088724a39", "name": "network-vif-plugged", "status": "completed", "tag": "7813289d-8139-4d10-ab8e-926e1b2e6577"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:23.390349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a1a88bb6-aff8-4918-bb19-4e0337606672 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1364 microversion: 2.1 time: 0.553293 Jul 24 01:01:23.391285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 44/89] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:22 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1364 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:23.394379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-6f12fa15-058a-4b30-af83-1aab012af940 req-83669aaf-5292-4cf7-b67c-0c3c907c4afe service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.395211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-6f12fa15-058a-4b30-af83-1aab012af940 req-83669aaf-5292-4cf7-b67c-0c3c907c4afe service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.399375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:23.400009 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:23.410441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.410921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.411711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.412304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.414583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-6f12fa15-058a-4b30-af83-1aab012af940 req-83669aaf-5292-4cf7-b67c-0c3c907c4afe service nova] Creating event network-vif-plugged:7813289d-8139-4d10-ab8e-926e1b2e6577 for instance 97d6538b-2585-40fc-b68c-e9a088724a39 on hv-cinder-maste Jul 24 01:01:23.420804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-6f12fa15-058a-4b30-af83-1aab012af940 req-83669aaf-5292-4cf7-b67c-0c3c907c4afe service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039857 Jul 24 01:01:23.422084 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 46/90] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:23.428984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:23.429687 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:23.436451 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.436898 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.437847 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.438576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.699729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.708426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0c3473f6-589b-421e-9ef4-bd9a62072fd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.315694 Jul 24 01:01:23.709015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 45/91] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:23 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:23.715233 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:23.715233 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:23.720261 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.720890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.721894 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:23.723143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:23.732690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.743965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f9f33062-4f26-4a01-b946-d4499c5cb4cb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.318154 Jul 24 01:01:23.753693 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 47/92] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:23 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:23.914934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:23.924664 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4a6cd0e5-6831-404b-a1fb-7a187f9c5766 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1501 microversion: 2.1 time: 0.214527 Jul 24 01:01:23.925616 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 46/93] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:23 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1501 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:24.424123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:24.425438 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:24.427459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:24.431103 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:24.448214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.450709 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.452528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.453134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.455179 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.456642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.457266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.457903 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.749000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:24.752783 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:24.757870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f1c3aeaa-dae6-414e-8536-21b905fd1a9b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1549 microversion: 2.1 time: 0.352546 Jul 24 01:01:24.758458 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 47/94] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:24 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1549 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:24.762957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:24.763563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:24.770556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.770760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:24.771054 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.771823 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.772262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.772414 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:24.773444 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-95352a94-c054-4276-960f-2f526b1cbafa tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.381155 Jul 24 01:01:24.774009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 48/95] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:24 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:24.778497 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:24.779126 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:24.785044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.785490 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.786255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:24.786690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:24.991387 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:24.995820 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:25.006992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-35bcc373-3dcf-4e52-aea3-15d1bab17bd9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.231834 Jul 24 01:01:25.007165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-24f3f403-888f-4454-a1a8-0ac0fc652d5f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.246895 Jul 24 01:01:25.016162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 48/96] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:24 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:25.016162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:25.016162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:25.017420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 49/97] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:24 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:25.020308 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:25.020765 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:25.021536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:25.021979 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:25.048768 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=100 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:25.140489 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-23594e75-7b91-406b-ae91-c3f1928bb024 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.131668 Jul 24 01:01:25.141091 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 49/98] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:01:25 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-interface => generated 284 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:25.370958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "accc7e85-bcaf-41cd-9682-ee665903899d"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:25.372140 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:25.378850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:25.379642 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:25.380419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:25.380851 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:25.445634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-74fd5312-cce1-4cea-b19f-026e5d459003 req-64a74167-4e14-4d21-a9f7-c6b013caf8fc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4389fd86-dd98-40ca-8e28-a9153502931c", "name": "network-vif-plugged", "status": "completed", "tag": "dd19007e-705d-4543-9981-8c458087a8e3"}, {"server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "name": "network-vif-plugged", "status": "completed", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:25.457205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-74fd5312-cce1-4cea-b19f-026e5d459003 req-64a74167-4e14-4d21-a9f7-c6b013caf8fc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:25.458184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-74fd5312-cce1-4cea-b19f-026e5d459003 req-64a74167-4e14-4d21-a9f7-c6b013caf8fc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:25.476491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-74fd5312-cce1-4cea-b19f-026e5d459003 req-64a74167-4e14-4d21-a9f7-c6b013caf8fc service nova] Creating event network-vif-plugged:dd19007e-705d-4543-9981-8c458087a8e3 for instance 4389fd86-dd98-40ca-8e28-a9153502931c on hv-cinder-maste Jul 24 01:01:25.476996 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-74fd5312-cce1-4cea-b19f-026e5d459003 req-64a74167-4e14-4d21-a9f7-c6b013caf8fc service nova] Creating event network-vif-plugged:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:01:25.484287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-74fd5312-cce1-4cea-b19f-026e5d459003 req-64a74167-4e14-4d21-a9f7-c6b013caf8fc service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.044832 Jul 24 01:01:25.484954 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 50/99] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:25 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:25.587460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:25.611211 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1656678926", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "networks": [{"uuid": "817ac8a4-2b25-4742-bdf4-9d6cc008d45a"}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:25.619888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:25.631760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=5 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:25.691983 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] validate_networks() for [('817ac8a4-2b25-4742-bdf4-9d6cc008d45a', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:01:26.535679 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-882fec32-a642-4eaa-88be-74c46b965f22 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "POST /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.169012 Jul 24 01:01:26.541668 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 50/100] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:01:25 2022] POST /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613/os-volume_attachments => generated 194 bytes in 1176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:26.547809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:26.548863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:26.557341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.557893 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.558911 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.559544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.656687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:01:26.657744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:01:26.658272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:01:26.658793 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:01:26.660370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:01:26.672596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.673165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.674142 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.674759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.703033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Getting quotas for project 4dd26bb2797345d6b531f5ba5db5cbc8. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:01:26.716132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Getting quotas for user 0e4b9bb864f24c93b2c9547432b8fd96 and project 4dd26bb2797345d6b531f5ba5db5cbc8. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:01:26.731963 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:01:26.735417 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:01:26.825725 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c5ea5232-e88f-4f0a-aea5-7b989b29e88c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.219900 Jul 24 01:01:26.826377 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 51/101] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:01:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 1222 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 24 01:01:26.835591 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:26.836212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:26.845664 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.846106 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.847498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.847944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.854457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:26.872071 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:26.873586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:26.874711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a6bf6f6c-ab0e-4867-8f39-4b1cff789bcf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.331746 Jul 24 01:01:26.875309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 51/102] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:26 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:26.891488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:26.892104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:26.899079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.899531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:26.900382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:26.901111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:27.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:27.369401 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a9d7d336-8795-419a-af1e-6533ec54b874 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.541693 Jul 24 01:01:27.379045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 52/103] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:26 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:27.379045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:27.379045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:27.385788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:27.386502 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:27.387506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:27.388116 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:27.417156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:27.435161 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a9b6b0da-9733-405c-bbb9-87e003ae1a68 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.547197 Jul 24 01:01:27.435762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 52/104] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:26 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:27.440145 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-3c899494-6e2b-47e5-8481-07f95d17c10e req-3909e7ab-7a2c-426f-ab85-854b41c8552e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64ba3b57-16f8-4b6f-b0a6-d0141afc0faa", "tag": "710ccda1-d8a7-4b65-9124-7d15ed6b5d75"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:27.450928 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3c899494-6e2b-47e5-8481-07f95d17c10e req-3909e7ab-7a2c-426f-ab85-854b41c8552e service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:27.451376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3c899494-6e2b-47e5-8481-07f95d17c10e req-3909e7ab-7a2c-426f-ab85-854b41c8552e service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:27.466127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-3c899494-6e2b-47e5-8481-07f95d17c10e req-3909e7ab-7a2c-426f-ab85-854b41c8552e service nova] Creating event network-changed:710ccda1-d8a7-4b65-9124-7d15ed6b5d75 for instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa on hv-cinder-maste Jul 24 01:01:27.470631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:27.472264 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-3c899494-6e2b-47e5-8481-07f95d17c10e req-3909e7ab-7a2c-426f-ab85-854b41c8552e service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035213 Jul 24 01:01:27.472874 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 53/105] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:27.485633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bc577c48-fa59-4eb4-a06c-58603b82995e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1223 microversion: 2.1 time: 0.114082 Jul 24 01:01:27.486791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 53/106] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:27 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1223 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:27.894647 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:27.895472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:27.907310 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:27.908025 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:27.909707 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:27.910331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.184606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:28.195849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:28.197894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:28.199860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-91e0f2c4-f5c9-43d3-ad20-f30e67598eb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.310796 Jul 24 01:01:28.201343 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 54/107] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:27 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:28.389911 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:28.390535 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:28.397289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:28.397747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.398590 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:28.398979 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.454036 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:28.454816 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:28.468860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:28.469759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.471165 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:28.472474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.624798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:28.642629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4f57f587-0e4f-4fc2-a770-93b0cfd5de65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.258937 Jul 24 01:01:28.650460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 54/108] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:28 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:28.655447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:28.659623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:28.670544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:28.675800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.678446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:28.679848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:28.745481 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:28.758010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-847a08b9-8150-4609-aad6-78b3e5731a57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1321 microversion: 2.1 time: 0.308180 Jul 24 01:01:28.758612 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 55/109] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:28 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1321 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:28.799109 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:28.810536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4a7b09e0-b1ef-441a-baab-f2040f3e8691 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1223 microversion: 2.1 time: 0.158560 Jul 24 01:01:28.811353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 55/110] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:28 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1223 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:29.219627 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:29.220329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:29.227878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.228592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.229380 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.230048 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.381725 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:29.390829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:29.392599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:29.394466 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f8a0116f-9cf8-47cf-acec-5d8d0c19c26a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1406 microversion: 2.20 time: 0.179301 Jul 24 01:01:29.395345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 56/111] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:29 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:01:29.667189 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:29.667803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:29.674444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.675388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.676236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.676710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.777386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:29.778010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:29.788829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.789281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.790314 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.791018 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.823224 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:29.832252 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9863c237-5454-4ce7-a10c-cd8c6a25e1a4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.169148 Jul 24 01:01:29.844990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 56/112] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:29 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:29.853937 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:29.854567 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:29.862729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.863309 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.864185 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:29.864852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:29.992360 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:30.002073 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d6540895-20ec-42b0-bffe-e339d0d2d197 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1215 microversion: 2.1 time: 0.155946 Jul 24 01:01:30.012165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 57/113] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:29 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1215 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:30.021479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:30.029864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9f61f8a3-2059-4b60-ac7d-4811d4e8f7a8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1515 microversion: 2.1 time: 0.258730 Jul 24 01:01:30.030450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 57/114] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:29 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1515 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:30.047900 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:30.049052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:30.060272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.060849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.064912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.065759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.096412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lazy-loading 'flavor' on Instance uuid 4389fd86-dd98-40ca-8e28-a9153502931c {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:01:30.223546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7372794c-0f69-4359-869c-c508b158edb6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c/action" status: 200 len: 32 microversion: 2.1 time: 0.179404 Jul 24 01:01:30.224117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 58/115] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:01:30 2022] POST /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c/action => generated 32 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:01:30.232984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:30.233591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:30.248813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.249255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.253419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.257164 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.416455 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:30.416759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:30.416759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:30.420552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-dc871546-c4c6-400a-9004-35f8272e5cf8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.191273 Jul 24 01:01:30.421138 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 58/116] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:30 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:30.423794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.425267 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.426947 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.427589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.575226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:30.583942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:30.585420 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:30.586646 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-de8c7e5a-9de1-4e9f-8ed0-e1f44051cfac tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1599 microversion: 2.20 time: 0.178388 Jul 24 01:01:30.595144 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 59/117] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:01:30 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1599 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:01:30.604650 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:30.605086 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:30.611175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.611723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.613121 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.613579 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.700387 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-86221a3b-9cf4-4e75-9f02-2dcc556befd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.099745 Jul 24 01:01:30.701003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 59/118] 10.222.0.18 () {60 vars in 1315 bytes} [Sun Jul 24 01:01:30 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-interface => generated 285 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 24 01:01:30.849834 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:30.851173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:30.858925 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.859866 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:30.861320 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:30.862399 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:31.020127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:31.020845 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:31.037905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:31.038368 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:31.039190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:31.039626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:31.237973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:31.251441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-74272457-cbcc-4bb9-b553-80c0f0a50502 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.405755 Jul 24 01:01:31.252318 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 60/119] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:30 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:31.266709 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:31.275098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a40a66f6-8e7a-460a-84f3-777823a2e15b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1215 microversion: 2.1 time: 0.259544 Jul 24 01:01:31.275700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 60/120] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:31 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1215 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:31.437589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:31.438209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:31.445375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:31.445827 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:31.446596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:31.447060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:31.580750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:31.599193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e3c7351c-80b1-4ca8-a543-abeba4df8125 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.165389 Jul 24 01:01:31.616644 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 61/121] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:31 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:32.269246 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:32.269944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:32.277072 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:32.277774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:32.278846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:32.279465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:32.284537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:32.292288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:32.292992 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:32.300815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:32.301681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:32.304917 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:32.309929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:32.459172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:32.473957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1271455d-b5d9-4c54-be8a-0e5ba8e43acc tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1277 microversion: 2.1 time: 0.182287 Jul 24 01:01:32.474591 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 62/122] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:32 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1277 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:32.573836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:32.586824 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b6e069ec-3217-4d5b-a74a-d7c731588bd7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.321272 Jul 24 01:01:32.587648 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 61/123] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:32 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:32.619152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:32.620118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:32.626549 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:32.627044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:32.628446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:32.629053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:32.767126 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:32.775554 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f3164948-dcaa-4c70-84b2-b1dff2c08c96 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.161256 Jul 24 01:01:32.776345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 63/124] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:32 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:32.990028 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-13bfdee8-eb13-4ce8-8ebc-d018dac22f2b req-c78efe0f-4b59-4c6f-becd-fa3742529ec9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c006a24-d0c5-4ab6-9ce4-259e05a9c56a", "name": "network-vif-plugged", "status": "completed", "tag": "3e10a0f3-72ea-499e-9aa7-f84921595c2e"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:32.999697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-13bfdee8-eb13-4ce8-8ebc-d018dac22f2b req-c78efe0f-4b59-4c6f-becd-fa3742529ec9 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.000283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-13bfdee8-eb13-4ce8-8ebc-d018dac22f2b req-c78efe0f-4b59-4c6f-becd-fa3742529ec9 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.013779 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-13bfdee8-eb13-4ce8-8ebc-d018dac22f2b req-c78efe0f-4b59-4c6f-becd-fa3742529ec9 service nova] Creating event network-vif-plugged:3e10a0f3-72ea-499e-9aa7-f84921595c2e for instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a on hv-cinder-maste Jul 24 01:01:33.019274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-13bfdee8-eb13-4ce8-8ebc-d018dac22f2b req-c78efe0f-4b59-4c6f-becd-fa3742529ec9 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033550 Jul 24 01:01:33.020283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 62/125] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:33.276817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-742dec20-ce77-4027-9472-8208a51a8943 req-b6447476-49ef-4f68-8854-da0921fb4389 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:33.292969 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-742dec20-ce77-4027-9472-8208a51a8943 req-b6447476-49ef-4f68-8854-da0921fb4389 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.294160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-742dec20-ce77-4027-9472-8208a51a8943 req-b6447476-49ef-4f68-8854-da0921fb4389 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.317087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-742dec20-ce77-4027-9472-8208a51a8943 req-b6447476-49ef-4f68-8854-da0921fb4389 service nova] Creating event network-changed:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:01:33.340441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-742dec20-ce77-4027-9472-8208a51a8943 req-b6447476-49ef-4f68-8854-da0921fb4389 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067168 Jul 24 01:01:33.343271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 64/126] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:33.500359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:33.501817 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:33.510487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.515443 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.516483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.517077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.607273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:33.613936 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:33.618523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:33.622548 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.623188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.624125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.624577 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.666526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:33.674813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-90c9dda0-f114-48e1-bced-139a57e46e6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.187222 Jul 24 01:01:33.675402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 63/127] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:33 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:33.766995 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:33.775281 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-167286a5-d5aa-4161-9bfc-65ada483c5f7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.174168 Jul 24 01:01:33.775866 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 65/128] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:33 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:33.793092 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:33.793710 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:33.799808 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.800256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.801041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:33.801478 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:33.930064 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:33.938108 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c6cf9373-c883-4626-81ad-ab06ec5000b5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.148903 Jul 24 01:01:33.938638 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 64/129] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:33 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:34.692139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:34.692684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:34.698964 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:34.699415 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:34.700168 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:34.700604 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:34.703131 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:34.792603 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:34.793250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:34.801702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:34.802158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:34.802979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:34.803421 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:34.837335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:34.844941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-252d340b-418b-4a57-81c4-bb77176f4181 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.157067 Jul 24 01:01:34.845669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 66/130] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:34 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:34.935998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:34.944229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b27d052d-163b-40e4-b579-7cec5921b3b4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.155485 Jul 24 01:01:34.945016 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 65/131] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:34 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:34.957484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:34.958092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:34.964213 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:34.964660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:34.965412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:34.965850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:35.101472 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:35.108965 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-60c1427f-7c7a-496c-a2e9-33ff4a7e35cd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.156068 Jul 24 01:01:35.109530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 67/132] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:34 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:35.861840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:35.862478 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:35.865582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:35.874318 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:35.874785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:35.875612 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:35.876053 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:35.961559 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:35.962278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:35.968992 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:35.969639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:35.970483 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:35.971086 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:36.007296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:36.014876 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9449ea1b-247a-4269-9397-5c888c4b2321 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.156948 Jul 24 01:01:36.015448 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 66/133] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:35 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:36.101207 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:36.108637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c6ccf296-0a95-44d4-9e3c-da4b05f6600c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.150968 Jul 24 01:01:36.109222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 68/134] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:35 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:36.125249 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:36.125790 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:36.132734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:36.133173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:36.133946 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:36.134374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:36.263612 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:36.271750 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-20a81dad-0ed7-4752-9ba8-7a6f7add5b26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.150381 Jul 24 01:01:36.272340 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 67/135] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:36 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:37.032393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:37.034277 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:37.040639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:37.041912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:37.048804 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.049309 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.051155 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.051932 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.125613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:37.126965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:37.136777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.137307 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.138164 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.138666 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.199287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:37.208689 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bf6bd432-fb69-48c2-af0f-8435d090d9a5 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.180168 Jul 24 01:01:37.209355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 69/136] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:37 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:37.243265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-360a7b55-ff91-4693-b003-8a503905cb78 req-2b0d8480-069b-4826-9210-56115000e8c2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6e7b2bf-8a97-487f-ae6d-fdfedd53b696", "tag": "7d0e4b10-307a-45ff-8fcf-b60161deca8f"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:37.252254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-360a7b55-ff91-4693-b003-8a503905cb78 req-2b0d8480-069b-4826-9210-56115000e8c2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.253103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-360a7b55-ff91-4693-b003-8a503905cb78 req-2b0d8480-069b-4826-9210-56115000e8c2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.274123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-360a7b55-ff91-4693-b003-8a503905cb78 req-2b0d8480-069b-4826-9210-56115000e8c2 service nova] Creating event network-changed:7d0e4b10-307a-45ff-8fcf-b60161deca8f for instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 on hv-cinder-maste Jul 24 01:01:37.279672 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-360a7b55-ff91-4693-b003-8a503905cb78 req-2b0d8480-069b-4826-9210-56115000e8c2 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041017 Jul 24 01:01:37.280846 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 70/137] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:37.289284 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:37.289976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:37.296253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.296691 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.297449 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:37.297882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:37.300307 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:37.308229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c6b28294-41dd-4a7c-ad74-32d985e46a1d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.186462 Jul 24 01:01:37.308813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 68/138] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:37 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:37.434793 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:37.443536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4e3d1262-30ec-46b0-8f45-7385a5ef73ed tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.158587 Jul 24 01:01:37.444432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 71/139] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:37 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:38.227243 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:38.227840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:38.234277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:38.234723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:38.235503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:38.235930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:38.325540 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:38.326256 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:38.332685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:38.333128 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:38.333887 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:38.334319 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:38.370927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:38.379450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e3bb8e94-a824-4cf9-a260-7c6f0cab9a56 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.156215 Jul 24 01:01:38.380036 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 69/140] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:38 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:38.461609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:38.462215 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:38.469733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:38.473521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:38.473979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:38.474789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:38.475242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:38.477443 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-423a0b42-a6eb-48e2-9efa-d4e76bad7944 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.155683 Jul 24 01:01:38.478198 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 72/141] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:38 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:38.614857 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:38.622858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-59ffe31f-f514-4629-acf9-1f65a72aa7fa tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.165421 Jul 24 01:01:38.623381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 70/142] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:38 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:39.396840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:39.397630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:39.404193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.404636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.406725 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.416051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.494360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:39.494996 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:39.508000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.508462 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.509643 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.510364 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.562034 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:39.573407 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e9e8702d-5707-4844-8596-936d2cc20f6d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.181096 Jul 24 01:01:39.574302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 73/143] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:39 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:39.640608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:39.641225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:39.650618 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.656087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.657289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.657747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.756453 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:39.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-56d75de1-666c-4381-b149-fdddfd98f51a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.274115 Jul 24 01:01:39.765553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 71/144] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:39 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:39.777280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:39.777772 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:39.784574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.785326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.786257 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.786873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.819817 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:01:39.837515 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:39.846724 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8a9bda0c-f914-4fb7-9c8c-9b02ec9cb647 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.209438 Jul 24 01:01:39.847700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 74/145] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:39 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:39.922051 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9b8f6af4-e9bb-466e-a436-98f8d46d056e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "DELETE /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 204 len: 0 microversion: 2.1 time: 0.155086 Jul 24 01:01:39.922617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 72/146] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:01:39 2022] DELETE /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:01:39.932769 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:39.933762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:39.940786 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.941435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:39.942206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:39.942732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:40.081439 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:40.089949 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9fcbd876-8baf-4082-9543-434bfac520fc tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1553 microversion: 2.1 time: 0.161961 Jul 24 01:01:40.090611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 75/147] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:39 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1553 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:40.595211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:40.597011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:40.606723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:40.607209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:40.608007 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:40.608599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:40.783225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:40.784753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:40.787800 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:40.796486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:40.797099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9e640dd9-2847-4770-a943-13a94a763a0d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.208910 Jul 24 01:01:40.797545 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:40.798554 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 73/148] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:40 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:40.798674 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:40.800783 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:40.865306 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:40.866164 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:40.872715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:40.873171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:40.873942 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:40.874619 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:41.001282 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:41.009515 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-dd6efb9f-8d26-45c6-9dd0-7d0912968861 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.230790 Jul 24 01:01:41.010560 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 76/149] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:40 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:41.046695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:41.062425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-36276caa-538b-46d1-ace3-c12bcfb90e57 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.201407 Jul 24 01:01:41.063053 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 74/150] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:40 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:41.108150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:41.108766 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:41.115751 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:41.116149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:41.116921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:41.117530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:41.254020 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:41.262296 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1afc2277-6bfe-459d-a314-32f7a6a44b3e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1553 microversion: 2.1 time: 0.157982 Jul 24 01:01:41.262918 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 77/151] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:41 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1553 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:41.814929 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:41.815703 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:41.823646 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:41.824203 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:41.825015 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:41.825488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:41.963181 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:41.970698 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d3b0e1f9-1a5b-48b7-9a22-97c8d1e579bb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.159568 Jul 24 01:01:41.971310 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 75/152] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:41 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:42.027385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:42.028012 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:42.034335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.034883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.035662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.036099 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.082007 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:42.082813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:42.089267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.089595 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.090460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.091028 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.230562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:42.240227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5461b6a0-b610-4e1e-b15f-0a46543cdd65 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1367 microversion: 2.1 time: 0.217034 Jul 24 01:01:42.241823 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 78/153] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:42 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1367 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:42.266666 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:42.271073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:42.276550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1a88761a-6a29-414a-b870-1c9ce08b5d10 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.198915 Jul 24 01:01:42.277374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 76/154] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:42 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:42.280814 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:42.281506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:42.288141 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.288714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.289510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.289947 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.436583 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:42.445716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8f715037-e99a-43f7-925e-9eeef52b240a tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1553 microversion: 2.1 time: 0.168977 Jul 24 01:01:42.446284 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 79/155] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:42 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1553 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:42.988454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:42.989006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:42.996426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.996879 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:42.997640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:42.998072 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.138627 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:43.146402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-feb11943-c982-4821-b0c2-5ef3d9f9e4ad tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.162454 Jul 24 01:01:43.147000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 77/156] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:42 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:43.259816 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:43.260517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:43.266793 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.267241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.267990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.268457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.293490 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:43.294410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:43.302244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.306054 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.307230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.307765 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.460240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:43.468817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4c52b719-fd42-4453-bd73-09386dbdc963 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1557 microversion: 2.1 time: 0.213834 Jul 24 01:01:43.481930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 80/157] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:43 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1557 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:43.489395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:43.490587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:43.497463 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.497923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.498035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:43.498697 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.499170 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.507495 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-897ed8d0-8300-411f-9642-7f54f07746ea tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.217769 Jul 24 01:01:43.508088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 78/158] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:43 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:43.512711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:43.513134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:43.524435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.525050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.525962 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:43.526515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:43.635118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1093c243-e56a-4a26-8595-821b86bf3db5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.125939 Jul 24 01:01:43.635749 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 79/159] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:01:43 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-interface => generated 284 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:43.680111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:43.688792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c1070d64-2ee1-4993-823e-11a3fe42adda tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1553 microversion: 2.1 time: 0.205061 Jul 24 01:01:43.693515 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 81/160] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:43 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1553 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:44.166473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:44.167179 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:44.173840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:44.174518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:44.176110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:44.176418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:44.439850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:44.448783 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-39b748fc-8ae1-4924-8d02-6def5ababdf0 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.289189 Jul 24 01:01:44.449465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 80/161] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:44 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:44.525973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:44.527362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:44.535247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:44.536018 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:44.536868 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:44.537442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:44.707574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:44.708483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:44.715077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:44.715683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:44.716466 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:44.716894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:44.725222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:44.734214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a61d4a82-c2b7-468b-8ef9-0f5ecc752f18 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.213061 Jul 24 01:01:44.745701 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 82/162] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:44 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:44.975442 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:44.983735 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1f6cfc17-1357-4bbb-8714-a6b3b724b7de tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1553 microversion: 2.1 time: 0.280841 Jul 24 01:01:44.984330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 81/163] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:44 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1553 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:45.009809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-bba5e113-4448-4fe5-a366-777b1c250487 req-f4f6d911-4196-400f-beda-a654ec06e2e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfcb8b81-6e4b-47a1-9b46-fa2088dbc613", "name": "network-vif-deleted", "tag": "53ce7301-1644-4424-b492-4f95801956b5"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:45.025190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-bba5e113-4448-4fe5-a366-777b1c250487 req-f4f6d911-4196-400f-beda-a654ec06e2e5 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:45.026224 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-bba5e113-4448-4fe5-a366-777b1c250487 req-f4f6d911-4196-400f-beda-a654ec06e2e5 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:45.041627 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-bba5e113-4448-4fe5-a366-777b1c250487 req-f4f6d911-4196-400f-beda-a654ec06e2e5 service nova] Creating event network-vif-deleted:53ce7301-1644-4424-b492-4f95801956b5 for instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 on hv-cinder-maste Jul 24 01:01:45.052665 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-bba5e113-4448-4fe5-a366-777b1c250487 req-f4f6d911-4196-400f-beda-a654ec06e2e5 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047089 Jul 24 01:01:45.053865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 83/164] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:45.468192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:45.468814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:45.482501 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:45.483178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:45.485089 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:45.486241 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:45.652591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:45.660758 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-6987da49-f01a-4c3e-8729-1a81e520873e req-bed21f37-b987-4138-b43f-505b6a67bccd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c006a24-d0c5-4ab6-9ce4-259e05a9c56a", "tag": "3e10a0f3-72ea-499e-9aa7-f84921595c2e"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:45.661781 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d8a48d72-fa70-4d8c-b450-b6c1dd0163aa tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.198134 Jul 24 01:01:45.662545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 82/165] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:45 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:45.671358 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-6987da49-f01a-4c3e-8729-1a81e520873e req-bed21f37-b987-4138-b43f-505b6a67bccd service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:45.671970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-6987da49-f01a-4c3e-8729-1a81e520873e req-bed21f37-b987-4138-b43f-505b6a67bccd service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:45.677493 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=100 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:45.687283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-6987da49-f01a-4c3e-8729-1a81e520873e req-bed21f37-b987-4138-b43f-505b6a67bccd service nova] Creating event network-changed:3e10a0f3-72ea-499e-9aa7-f84921595c2e for instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a on hv-cinder-maste Jul 24 01:01:45.693445 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-6987da49-f01a-4c3e-8729-1a81e520873e req-bed21f37-b987-4138-b43f-505b6a67bccd service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039650 Jul 24 01:01:45.695090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 84/166] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:01:45.751637 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:45.752240 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:45.759275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:45.759776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:45.760601 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:45.761032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:46.003232 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:46.004302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:46.017027 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:46.017669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:46.018496 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:46.020466 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:46.165349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:46.168930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:46.173543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-24dae874-77be-4c4e-9265-75d1c2dc95fe tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.425778 Jul 24 01:01:46.175392 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 83/167] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:45 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:46.187138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5abd5ec6-92b1-4225-9eaa-640430229837 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1511 microversion: 2.1 time: 0.188850 Jul 24 01:01:46.187723 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 85/168] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:45 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1511 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:46.679615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:46.680208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:46.686273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:46.686718 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:46.687538 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:46.687965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:46.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:46.846429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-718beb49-8ba3-4c43-af42-2ebd0e30c238 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.171219 Jul 24 01:01:46.847075 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 84/169] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:46 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:47.193385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:47.194545 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:47.200918 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.201715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:47.202484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.203035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:47.212320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:47.220836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:47.244464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.245303 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:47.246323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.247032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:47.314460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.314954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:47.315561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lazy-loading 'fault' on Instance uuid dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:01:47.375687 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b88e33e967ed47e780b3a1d64c5a0b8a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c129535452294d998fee4c9e259c9d41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:47.377327 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:47.387586 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-46a07fe9-c54a-43a7-b3b4-c682e9697469 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.198576 Jul 24 01:01:47.388378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 86/170] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:47 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:47.390237 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-60bf9ba3-5d10-4a94-986f-f74f6fae97a5 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 200 len: 1516 microversion: 2.1 time: 0.188618 Jul 24 01:01:47.390845 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 85/171] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:47 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 1516 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:47.863880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:47.864697 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:47.870837 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.871360 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:47.872308 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:47.872767 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.102123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:48.109916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4b5b7124-744b-422f-a79b-3a0601974523 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.250100 Jul 24 01:01:48.110392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 87/172] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:47 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:48.413979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:48.414598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:48.420579 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:48.421282 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:48.427087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:48.427569 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.428215 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:48.428511 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:48.428945 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.429077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.429914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:48.430455 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.448954 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] HTTP exception thrown: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Request method failure captured: Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 HTTP/1.1 Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm6wGlWI8gF-0DQ4386AXrVJFgv4xjpC9iLZY1mzFY7vTXzvQn5PdnMpEyWO22n3DYjNIot5sYGroPC1jk4XcYcbXdVJwHUGDOqqmNY-FcJUM6xdWCR4sYe_eWrkmKAtlUYQFqdHzbg0oK1RkGzPX1J4mJvsyJlteZHmL85SSbpWpI Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:48.454274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/c129535452294d998fee4c9e259c9d41"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_c129535452294d998fee4c9e259c9d41", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: b88e33e967ed47e780b3a1d64c5a0b8a Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.456362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'dfcb8b81-6e4b-47a1-9b46-fa2088dbc613'} Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:48.457032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.458130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.460679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Returning 404 to user: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:48.461822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-123d9ab9-c10f-4469-943e-9ca058992f8e tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 404 len: 111 microversion: 2.1 time: 0.050644 Jul 24 01:01:48.462922 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 88/173] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:48 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:01:48.598920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:48.605409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:48.606352 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:48.607908 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fd96178a-36a4-4abf-8883-0bb501c07b03 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.199012 Jul 24 01:01:48.608701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 86/174] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:48 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:48.612822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:48.613274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.614056 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:48.614503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:48.628811 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] HTTP exception thrown: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Request method failure captured: Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613/os-volume_attachments HTTP/1.1 Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm6wGlWI8gF-0DQ4386AXrVJFgv4xjpC9iLZY1mzFY7vTXzvQn5PdnMpEyWO22n3DYjNIot5sYGroPC1jk4XcYcbXdVJwHUGDOqqmNY-FcJUM6xdWCR4sYe_eWrkmKAtlUYQFqdHzbg0oK1RkGzPX1J4mJvsyJlteZHmL85SSbpWpI Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:48.630641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/c129535452294d998fee4c9e259c9d41"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_c129535452294d998fee4c9e259c9d41", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: b88e33e967ed47e780b3a1d64c5a0b8a Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.632097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'server_id': 'dfcb8b81-6e4b-47a1-9b46-fa2088dbc613'} Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:48.632927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:48.634305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:48.635521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Returning 404 to user: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:48.635521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7836cf84-c91f-426e-b0bc-86859cb78ef3 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.031038 Jul 24 01:01:48.635521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 89/175] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:01:48 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613/os-volume_attachments => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:01:49.133674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:49.134436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:49.141204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.141826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.142905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.144215 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.289001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:49.296277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f22b74d3-f5ba-4298-b14d-b2969f9d7447 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.166613 Jul 24 01:01:49.296842 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 87/176] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:49 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:49.358731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-8dc13056-7f13-4a63-8055-884356a5a7bc req-c9910017-c52c-4b47-81df-9fe6223120ed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6e7b2bf-8a97-487f-ae6d-fdfedd53b696", "name": "network-vif-plugged", "status": "completed", "tag": "7d0e4b10-307a-45ff-8fcf-b60161deca8f"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:49.367102 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-8dc13056-7f13-4a63-8055-884356a5a7bc req-c9910017-c52c-4b47-81df-9fe6223120ed service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.367546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-8dc13056-7f13-4a63-8055-884356a5a7bc req-c9910017-c52c-4b47-81df-9fe6223120ed service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.379480 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-8dc13056-7f13-4a63-8055-884356a5a7bc req-c9910017-c52c-4b47-81df-9fe6223120ed service nova] Creating event network-vif-plugged:7d0e4b10-307a-45ff-8fcf-b60161deca8f for instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 on hv-cinder-maste Jul 24 01:01:49.384445 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-8dc13056-7f13-4a63-8055-884356a5a7bc req-c9910017-c52c-4b47-81df-9fe6223120ed service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029297 Jul 24 01:01:49.385097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 90/177] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:01:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:01:49.624873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:49.626301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:49.634099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.634743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.635716 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.636372 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.774169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:49.781863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-043dfe72-d082-41f4-930e-c1a11033dcd0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.160814 Jul 24 01:01:49.782515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 88/178] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:49 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:49.828160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:49.828776 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:49.835880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.836330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.837091 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.837529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.851491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] HTTP exception thrown: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Request method failure captured: Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 HTTP/1.1 Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm6wGlWI8gF-0DQ4386AXrVJFgv4xjpC9iLZY1mzFY7vTXzvQn5PdnMpEyWO22n3DYjNIot5sYGroPC1jk4XcYcbXdVJwHUGDOqqmNY-FcJUM6xdWCR4sYe_eWrkmKAtlUYQFqdHzbg0oK1RkGzPX1J4mJvsyJlteZHmL85SSbpWpI Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:49.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/c129535452294d998fee4c9e259c9d41"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_c129535452294d998fee4c9e259c9d41", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: b88e33e967ed47e780b3a1d64c5a0b8a Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.857398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'dfcb8b81-6e4b-47a1-9b46-fa2088dbc613'} Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:49.858173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.859257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.860372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Returning 404 to user: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:49.860372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-db9590d7-4961-43cf-9858-cbec941aec6b tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "DELETE /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 404 len: 111 microversion: 2.1 time: 0.033617 Jul 24 01:01:49.860372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 91/179] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:01:49 2022] DELETE /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:01:49.867062 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:49.867864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] [instance: dfcb8b81-6e4b-47a1-9b46-fa2088dbc613] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:49.874115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.874589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.875571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:49.876098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:49.890575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] HTTP exception thrown: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Request method failure captured: Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 HTTP/1.1 Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm6wGlWI8gF-0DQ4386AXrVJFgv4xjpC9iLZY1mzFY7vTXzvQn5PdnMpEyWO22n3DYjNIot5sYGroPC1jk4XcYcbXdVJwHUGDOqqmNY-FcJUM6xdWCR4sYe_eWrkmKAtlUYQFqdHzbg0oK1RkGzPX1J4mJvsyJlteZHmL85SSbpWpI Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:01:49.895849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/c129535452294d998fee4c9e259c9d41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/c129535452294d998fee4c9e259c9d41"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_c129535452294d998fee4c9e259c9d41", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: c129535452294d998fee4c9e259c9d41 Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-DeleteServersTestJSON-653386294 Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: b88e33e967ed47e780b3a1d64c5a0b8a Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-DeleteServersTestJSON-653386294-project Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.897345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'dfcb8b81-6e4b-47a1-9b46-fa2088dbc613'} Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:49.897978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. Jul 24 01:01:49.899349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:49.900717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] Returning 404 to user: Instance dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:49.900717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7af4e1f4-9778-4cbc-be1d-b8bca5261947 tempest-DeleteServersTestJSON-653386294 tempest-DeleteServersTestJSON-653386294-project] 10.222.0.18 "GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613" status: 404 len: 111 microversion: 2.1 time: 0.035134 Jul 24 01:01:49.900717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 89/180] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:49 2022] GET /compute/v2.1/servers/dfcb8b81-6e4b-47a1-9b46-fa2088dbc613 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:01:50.314322 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:50.315539 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:50.323209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:50.323651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:50.324566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:50.325069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:50.613380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:50.624671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7ee67a78-d7ab-433c-8249-69a4ae12f257 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1319 microversion: 2.1 time: 0.314595 Jul 24 01:01:50.625267 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 92/181] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:50 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1319 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:50.800478 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:50.801100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:50.807740 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:50.808192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:50.809000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:50.809434 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:51.004559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:51.012732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f2891964-055f-4025-98fb-38b4bd644d19 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.217515 Jul 24 01:01:51.013316 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 90/182] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:50 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:51.649830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:51.650452 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:51.661904 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:51.662350 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:51.663161 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:51.663609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:52.032044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:52.032654 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:52.039647 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:52.040321 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:52.041112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:52.041676 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:52.246046 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:52.253951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-34d7bb3a-3d43-49db-88aa-43d4a630264f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1506 microversion: 2.1 time: 0.608048 Jul 24 01:01:52.254476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 93/183] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:51 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1506 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:52.325979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:52.336317 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ebb8e76c-27a8-464f-aee9-889cee48c4ba tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.308576 Jul 24 01:01:52.338626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 91/184] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:52 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:53.357514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:53.359561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:53.382636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:53.383600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:53.384564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:53.385179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:53.560889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:53.575896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9832658a-7994-4280-9262-ec6c9bf82d4c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.224817 Jul 24 01:01:53.577251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 94/185] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:53 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:54.594297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:54.594894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:54.601977 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:54.602973 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:54.603873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:54.604618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:54.746726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:54.756295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4fb6a6e8-a5d6-4ea1-8d24-d8d696cbf10c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.165979 Jul 24 01:01:54.758662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 92/186] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:54 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:55.775418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:55.776018 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:55.782096 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:55.782554 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:55.783353 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:55.784045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:55.871475 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bc7638fb-5a11-4e61-b177-0c0663ed2665 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Action: 'create', calling method: .version_select at 0x7f5e308e38b0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1439632708", "metadata": {"Type": "work"}}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bc7638fb-5a11-4e61-b177-0c0663ed2665 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Request method failure captured: Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 122 Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JoDNXBu5sVTtex21aY6URB5OjgO0nLKr5oL6SVX8hhDT5Vh3Gu3grpRECCZCdkxxR5C-YV7ELE0fHQggyosf0CY1o_qBOUI_0ffNlqZ-gA66ftyfAjXf_KcqWzMTPdhxexnhRw9sz-BGf6PVlut02VK0ojz4e5PBNgVaF14yutIeno Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:55.887188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75686cdd2e114839b8ecf5a4fc1a2dbe", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 1cc2bcd94bdc447bb522e059edd8d49e Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1439632708", "metadata": {"Type": "work"}}} Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:01:55.888631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1439632708', 'metadata': {'Type': 'work'}}}} Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise error Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi '#$%' Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:55.889476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 24 01:01:55.890796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 24 01:01:55.891960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 24 01:01:55.891960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.891960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bc7638fb-5a11-4e61-b177-0c0663ed2665 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:55.891960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bc7638fb-5a11-4e61-b177-0c0663ed2665 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.142619 Jul 24 01:01:55.891960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 95/187] 10.222.0.18 () {60 vars in 1155 bytes} [Sun Jul 24 01:01:55 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 144 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 24 01:01:55.899454 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-30540992-1f64-423e-8839-c8e1acffa05b tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Action: 'create', calling method: .version_select at 0x7f5e306a6e50>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1625915001", "metadata": {"Type": "work"}}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-30540992-1f64-423e-8839-c8e1acffa05b tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Request method failure captured: Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 120 Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JoDNXBu5sVTtex21aY6URB5OjgO0nLKr5oL6SVX8hhDT5Vh3Gu3grpRECCZCdkxxR5C-YV7ELE0fHQggyosf0CY1o_qBOUI_0ffNlqZ-gA66ftyfAjXf_KcqWzMTPdhxexnhRw9sz-BGf6PVlut02VK0ojz4e5PBNgVaF14yutIeno Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:55.901800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75686cdd2e114839b8ecf5a4fc1a2dbe", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 1cc2bcd94bdc447bb522e059edd8d49e Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1625915001", "metadata": {"Type": "work"}}} Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:01:55.903317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1625915001', 'metadata': {'Type': 'work'}}}} Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise error Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi '0' Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:55.904093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 24 01:01:55.905396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 24 01:01:55.906609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 24 01:01:55.906609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.906609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-30540992-1f64-423e-8839-c8e1acffa05b tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:55.906609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-30540992-1f64-423e-8839-c8e1acffa05b tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.007563 Jul 24 01:01:55.906609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 96/188] 10.222.0.18 () {60 vars in 1155 bytes} [Sun Jul 24 01:01:55 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 24 01:01:55.915669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3e3ab41c-bd72-4b55-a3b2-a785b9439b73 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Action: 'create', calling method: .version_select at 0x7f5e308e3ca0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1753737170", "metadata": {"Type": "work"}}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3e3ab41c-bd72-4b55-a3b2-a785b9439b73 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Request method failure captured: Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 119 Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JoDNXBu5sVTtex21aY6URB5OjgO0nLKr5oL6SVX8hhDT5Vh3Gu3grpRECCZCdkxxR5C-YV7ELE0fHQggyosf0CY1o_qBOUI_0ffNlqZ-gA66ftyfAjXf_KcqWzMTPdhxexnhRw9sz-BGf6PVlut02VK0ojz4e5PBNgVaF14yutIeno Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:55.917956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75686cdd2e114839b8ecf5a4fc1a2dbe", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 1cc2bcd94bdc447bb522e059edd8d49e Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1753737170", "metadata": {"Type": "work"}}} Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:01:55.919486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1753737170', 'metadata': {'Type': 'work'}}}} Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise error Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi '' Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:55.920367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 24 01:01:55.921858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 24 01:01:55.922994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 24 01:01:55.922994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.922994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3e3ab41c-bd72-4b55-a3b2-a785b9439b73 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:55.922994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3e3ab41c-bd72-4b55-a3b2-a785b9439b73 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.008387 Jul 24 01:01:55.923711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 97/189] 10.222.0.18 () {60 vars in 1155 bytes} [Sun Jul 24 01:01:55 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 13 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 24 01:01:55.927045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:55.930464 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-87329840-7135-42a8-be7c-701e0603d7a9 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Calling method '.version_select at 0x7f5e308bf3a0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:55.935332 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0bbf5178-381c-4582-9cb5-ee4196e20d06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.164061 Jul 24 01:01:55.936280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 93/190] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:55 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:01:55.964945 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-87329840-7135-42a8-be7c-701e0603d7a9 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] HTTP exception thrown: Volume tempest-invalid-517890872 could not be found. Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-87329840-7135-42a8-be7c-701e0603d7a9 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Request method failure captured: Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-517890872 HTTP/1.1 Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JoDNXBu5sVTtex21aY6URB5OjgO0nLKr5oL6SVX8hhDT5Vh3Gu3grpRECCZCdkxxR5C-YV7ELE0fHQggyosf0CY1o_qBOUI_0ffNlqZ-gA66ftyfAjXf_KcqWzMTPdhxexnhRw9sz-BGf6PVlut02VK0ojz4e5PBNgVaF14yutIeno Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:55.967011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75686cdd2e114839b8ecf5a4fc1a2dbe", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 1cc2bcd94bdc447bb522e059edd8d49e Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Volume tempest-invalid-517890872 could not be found. Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'tempest-invalid-517890872'} Jul 24 01:01:55.968409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-517890872 could not be found. (HTTP 404) (Request-ID: req-1cd0ac2f-be45-439d-b72d-f288b8704d83) Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.969193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 24 01:01:55.970435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-517890872 could not be found. Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:55.971733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-517890872 could not be found. Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-517890872 could not be found. Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-87329840-7135-42a8-be7c-701e0603d7a9 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Returning 404 to user: Volume tempest-invalid-517890872 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-87329840-7135-42a8-be7c-701e0603d7a9 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "DELETE /compute/v2.1/os-volumes/tempest-invalid-517890872" status: 404 len: 98 microversion: 2.1 time: 0.041512 Jul 24 01:01:55.973124 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 98/191] 10.222.0.18 () {58 vars in 1214 bytes} [Sun Jul 24 01:01:55 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-517890872 => generated 98 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 24 01:01:55.979023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-973aca1e-dc34-4f4e-ae93-6859c7fc7f8e tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003825 Jul 24 01:01:55.979530 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 94/192] 10.222.0.18 () {58 vars in 1139 bytes} [Sun Jul 24 01:01:55 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 24 01:01:55.989282 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-96bb2400-105e-49c5-ace2-4cab1e4f587d tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004932 Jul 24 01:01:55.989865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 99/193] 10.222.0.18 () {58 vars in 1136 bytes} [Sun Jul 24 01:01:55 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 24 01:01:55.999362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-96c4bffe-fc22-4076-a0c0-1696ae41fa76 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Calling method '.version_select at 0x7f5e306e5670>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:56.032556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-96c4bffe-fc22-4076-a0c0-1696ae41fa76 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] HTTP exception thrown: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-96c4bffe-fc22-4076-a0c0-1696ae41fa76 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Request method failure captured: Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: DELETE /compute/v2.1/os-volumes/953462e0-2292-48b8-a8c3-877819086978 HTTP/1.1 Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JoDNXBu5sVTtex21aY6URB5OjgO0nLKr5oL6SVX8hhDT5Vh3Gu3grpRECCZCdkxxR5C-YV7ELE0fHQggyosf0CY1o_qBOUI_0ffNlqZ-gA66ftyfAjXf_KcqWzMTPdhxexnhRw9sz-BGf6PVlut02VK0ojz4e5PBNgVaF14yutIeno Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:01:56.036274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75686cdd2e114839b8ecf5a4fc1a2dbe", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 1cc2bcd94bdc447bb522e059edd8d49e Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '953462e0-2292-48b8-a8c3-877819086978'} Jul 24 01:01:56.037649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. (HTTP 404) (Request-ID: req-26b82724-e948-4c84-9508-4f4f001e2e53) Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:56.038514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 24 01:01:56.039536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 24 01:01:56.040496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-96c4bffe-fc22-4076-a0c0-1696ae41fa76 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Returning 404 to user: Volume 953462e0-2292-48b8-a8c3-877819086978 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-96c4bffe-fc22-4076-a0c0-1696ae41fa76 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "DELETE /compute/v2.1/os-volumes/953462e0-2292-48b8-a8c3-877819086978" status: 404 len: 109 microversion: 2.1 time: 0.042057 Jul 24 01:01:56.041536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 95/194] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:01:55 2022] DELETE /compute/v2.1/os-volumes/953462e0-2292-48b8-a8c3-877819086978 => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:01:56.047335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e3ed10aa-6b76-4ec0-8f57-1346872164c4 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Calling method '.version_select at 0x7f5e3068b8b0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:56.075962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-e3ed10aa-6b76-4ec0-8f57-1346872164c4 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] HTTP exception thrown: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e3ed10aa-6b76-4ec0-8f57-1346872164c4 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Request method failure captured: Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/os-volumes/9574ef5b-6a57-4f45-b123-3fbafe3404b1 HTTP/1.1 Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JoDNXBu5sVTtex21aY6URB5OjgO0nLKr5oL6SVX8hhDT5Vh3Gu3grpRECCZCdkxxR5C-YV7ELE0fHQggyosf0CY1o_qBOUI_0ffNlqZ-gA66ftyfAjXf_KcqWzMTPdhxexnhRw9sz-BGf6PVlut02VK0ojz4e5PBNgVaF14yutIeno Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:01:56.077910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75686cdd2e114839b8ecf5a4fc1a2dbe"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75686cdd2e114839b8ecf5a4fc1a2dbe", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75686cdd2e114839b8ecf5a4fc1a2dbe Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesNegativeTest-1782628291 Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 1cc2bcd94bdc447bb522e059edd8d49e Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesNegativeTest-1782628291-project Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '9574ef5b-6a57-4f45-b123-3fbafe3404b1'} Jul 24 01:01:56.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. (HTTP 404) (Request-ID: req-394f9605-b12a-421b-8e79-88ec57a8daeb) Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:56.080153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:01:56.081909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 24 01:01:56.083543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e3ed10aa-6b76-4ec0-8f57-1346872164c4 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] Returning 404 to user: Volume 9574ef5b-6a57-4f45-b123-3fbafe3404b1 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e3ed10aa-6b76-4ec0-8f57-1346872164c4 tempest-VolumesNegativeTest-1782628291 tempest-VolumesNegativeTest-1782628291-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/9574ef5b-6a57-4f45-b123-3fbafe3404b1" status: 404 len: 109 microversion: 2.1 time: 0.035776 Jul 24 01:01:56.084959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 100/195] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:01:56 2022] GET /compute/v2.1/os-volumes/9574ef5b-6a57-4f45-b123-3fbafe3404b1 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:01:56.955265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:56.956770 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:56.963192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:56.963723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:56.964579 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:56.965092 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:56.969573 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=100 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:01:57.130728 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:57.140301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a1a53aca-395d-49f2-8858-c3cf09a31dfc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.190399 Jul 24 01:01:57.140881 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 96/196] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:56 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:58.166798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:58.167411 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:58.174803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:58.175434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:58.176443 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:58.177068 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:58.378243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:58.391788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-94e3cc0f-468f-4abd-a3a1-a721b353993a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.229388 Jul 24 01:01:58.392669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 101/197] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:58 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:01:59.409780 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:01:59.410383 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:01:59.417115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:59.417565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:59.418327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:01:59.418759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:01:59.544497 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:01:59.552168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4506582a-8017-46b6-ac7a-ab7e8028e8c8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.146600 Jul 24 01:01:59.552733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 97/198] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:01:59 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:00.569727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:00.570362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:00.576475 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:00.576926 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:00.577691 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:00.578127 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:00.708787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:00.716261 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-43460848-36d8-4959-9fb8-29df277fe29d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.151128 Jul 24 01:02:00.716821 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 102/199] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:00 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:01.733699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:01.734301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:01.741479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:01.741927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:01.742714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:01.743175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:01.876565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:01.884430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-822d63a4-31d4-4e44-a1b3-291aba2bad3a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.154650 Jul 24 01:02:01.884996 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 98/200] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:01 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:02.902663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:02.903237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:02.910050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:02.910495 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:02.911280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:02.911714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:03.042074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:03.049460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-095e07c5-ceb7-4e22-9c91-0bcde146ecc6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.150793 Jul 24 01:02:03.050225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 103/201] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:02 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:04.066888 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:04.067534 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:04.074349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:04.074826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:04.075593 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:04.076029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:04.211213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:04.218591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7663d529-4bf3-47ad-bb2c-79f9ea6e9373 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.156123 Jul 24 01:02:04.219188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 99/202] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:04 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:05.238280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:05.239475 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:05.242675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=66 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:05.247130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:05.247759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:05.248547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:05.248993 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:05.407977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:05.419232 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-25b98239-fac0-437d-b250-f07e65e37d26 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1521 microversion: 2.1 time: 0.186314 Jul 24 01:02:05.420001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 104/203] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:05 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1521 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:06.437590 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:06.438206 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:06.442453 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:06.446465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:06.446982 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:06.447929 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:06.448372 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:06.632963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:06.640999 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-330db45b-5d4a-4bf6-b6db-fd096a1f802e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1501 microversion: 2.1 time: 0.207367 Jul 24 01:02:06.641585 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 100/204] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:06 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1501 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:06.657196 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:06.657803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:06.664088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:06.665215 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:06.666440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:06.667042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:06.861490 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:06.869691 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-af1704d6-9096-49ef-b902-3cea41b733e7 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1514 microversion: 2.1 time: 0.216167 Jul 24 01:02:06.870428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 105/205] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:06 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1514 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:08.392409 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:08.394232 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:08.406081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:08.406571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:08.407599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:08.407905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:08.434752 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lazy-loading 'flavor' on Instance uuid 97d6538b-2585-40fc-b68c-e9a088724a39 {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:02:08.558744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-cbc6693c-6da5-4a8a-b2fb-adfad19e7072 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/action" status: 200 len: 32 microversion: 2.1 time: 0.170357 Jul 24 01:02:08.559376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 101/206] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:02:08 2022] POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/action => generated 32 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:02:08.569336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:08.569951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:08.576153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:08.576603 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:08.577378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:08.577818 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:08.725520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:08.733834 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-68046cbc-cad6-4a9c-8fef-e6b10d049367 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.169123 Jul 24 01:02:08.734423 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 106/207] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:08 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:09.754617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:09.755423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:09.762014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:09.762471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:09.763287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:09.763824 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:10.059391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:10.068651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d4e274e4-3cda-40c2-8d2f-f6878875a29e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.320188 Jul 24 01:02:10.069403 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 102/208] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:09 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:11.085199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:11.085811 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:11.092192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:11.092642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:11.093473 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:11.093999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:11.255007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:11.262982 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ed340967-7ca5-4d23-93fc-01a9a92a80b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.181424 Jul 24 01:02:11.263840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 107/209] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:11 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:12.280353 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:12.281726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:12.288407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:12.288853 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:12.289931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:12.290356 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:12.423348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:12.431584 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-79c2699e-07cf-4848-8d7f-c15a4772f748 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.155148 Jul 24 01:02:12.432228 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 103/210] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:12 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:13.450687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:13.451516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:13.459178 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:13.459751 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:13.460719 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:13.461250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:13.681206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:13.689705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-32870b4c-e41c-42bc-a879-8da7fce1ecce tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.243167 Jul 24 01:02:13.690370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 108/211] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:13 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:14.708414 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:14.710294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:14.716935 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:14.717603 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:14.718379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:14.718836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:14.862935 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:14.872776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f65f3d70-29f3-420b-a439-9370c233f175 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.169240 Jul 24 01:02:14.873580 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 104/212] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:14 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:15.890101 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:15.890715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:15.897498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:15.897950 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:15.898718 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:15.899183 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:15.903181 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=94 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:16.032287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:16.040004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a3602a60-fc5f-43f6-a9e2-1ab3f4ce8712 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.153781 Jul 24 01:02:16.040569 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 109/213] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:15 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:17.057983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:17.058593 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:17.065323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:17.065767 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:17.066518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:17.066984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:17.208972 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:17.218231 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fd4b919f-fcb3-484a-a2bc-6228e7cd99d1 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.163964 Jul 24 01:02:17.218971 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 105/214] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:17 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:18.236743 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:18.237355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:18.243412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:18.243847 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:18.244606 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:18.245040 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:18.384573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:18.393807 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bcfe57dc-1493-4809-82d3-25516b6f498a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.161121 Jul 24 01:02:18.395146 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 110/215] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:18 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:19.385216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8ef10991-dc64-4e53-aab2-c929ece8ecbb", "device": "/dev/sdb"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:19.386812 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:19.393079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:19.393671 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:19.394656 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:19.395115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:19.411752 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:19.412920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:19.419118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:19.419562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:19.420320 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:19.420917 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:19.568273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:19.578239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7caa6437-3720-42a9-800b-3d452570db24 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.170730 Jul 24 01:02:19.582823 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 111/216] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:19 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:20.595722 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:20.596441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:20.602825 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:20.603268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:20.604018 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:20.604444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:20.731218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:20.739176 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-49f44e4d-7b05-4340-bd4c-c647cf1f7683 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.147305 Jul 24 01:02:20.739753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 112/217] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:20 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:21.757720 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:21.758557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:21.765894 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:21.766340 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:21.767131 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:21.767562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:21.894282 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:21.902313 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-68f5b28c-0ed5-4f08-86e6-f7a8639da05a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.150208 Jul 24 01:02:21.902907 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 113/218] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:21 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:22.229452 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-28771b87-0e68-4a7a-8b39-4e7280c0882a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "POST /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.847919 Jul 24 01:02:22.230063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 106/219] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:02:19 2022] POST /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696/os-volume_attachments => generated 194 bytes in 2849 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:22.918676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:22.919395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:22.926696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:22.927183 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:22.927959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:22.928398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:23.062897 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:23.071711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d7e386f7-58cc-4817-b530-49b7d292a4a0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.157098 Jul 24 01:02:23.072274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 114/220] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:22 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:24.089138 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:24.089986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:24.097252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:24.097701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:24.098473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:24.098934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:24.248353 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:24.256639 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-45811d1a-f884-4385-a1b7-4b2b38c8c169 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.171535 Jul 24 01:02:24.257597 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 107/221] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:24 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:25.273382 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:25.273987 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:25.280034 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:25.280484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:25.281243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:25.281683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:25.415899 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:25.425404 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fc546bb0-2079-4036-ba29-2b2ef6ecf363 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.155810 Jul 24 01:02:25.426067 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 115/222] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:25 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:26.311109 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:26.311596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:26.318652 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:26.319382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:26.320149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:26.320575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:26.344918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-cda772a0-79f4-499d-998c-2aaddedff0ef tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/ips" status: 200 len: 145 microversion: 2.1 time: 0.037632 Jul 24 01:02:26.345579 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 108/223] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:02:26 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/ips => generated 145 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:26.442558 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:26.443202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:26.449371 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:26.449817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:26.450578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:26.451039 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:26.588253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:26.596581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bf8a22b9-e16b-40d6-b7ba-995896adf1c9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.157679 Jul 24 01:02:26.597165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 116/224] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:26 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:27.614542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:27.615191 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:27.621669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:27.622118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:27.622903 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:27.623332 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:27.627869 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:27.765171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:27.774087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-25ebc6b4-009c-4128-b746-cca25c48b3ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.164080 Jul 24 01:02:27.774786 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 109/225] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:27 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:28.796130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:28.796740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:28.803391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:28.803841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:28.804608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:28.805074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:28.939217 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:28.947493 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f37edb3f-d64b-4ed8-8634-337af4e4ae44 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.155222 Jul 24 01:02:28.948066 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 117/226] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:28 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:29.964793 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:29.965326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:29.972388 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:29.972833 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:29.973608 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:29.974040 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.104246 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:30.112653 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6d92d08c-d474-4af3-a751-15de5c52dffb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1500 microversion: 2.1 time: 0.151835 Jul 24 01:02:30.113258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 110/227] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:29 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1500 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:30.129016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b8661986-db4e-4719-956e-d387648adac9"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:30.130192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:30.136272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.136717 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.137474 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.137908 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.161685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] HTTP exception thrown: Cannot 'attach_volume' instance 97d6538b-2585-40fc-b68c-e9a088724a39 while it is in vm_state rescued Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Request method failure captured: Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/os-volume_attachments HTTP/1.1 Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 74 Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnFiejBFZa8mJ5D502lXi0vI2AJZSXzLQ6yvLVNBpg5wuvRciV8GsNNEfmF5OiwQV_hyTlbh1vvTV_RBBoRfFvSvMn5HPevJqiFZ0JTvv9BtvBk2xvVAY9_4RTM-tU7gCIt6rWRh-Ok03kE5HtQWgevEOi1XOD-L9IgzsF_0g4DZ8U Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:02:30.163269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_f7d219e82efe44f9b6f860a5fabc7733", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 283a1bfe0e3b45aaa0a33e5fe6b1d85a Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {"volumeAttachment": {"volumeId": "b8661986-db4e-4719-956e-d387648adac9"}} Jul 24 01:02:30.164745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Cannot 'attach_volume' instance 97d6538b-2585-40fc-b68c-e9a088724a39 while it is in vm_state rescued Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'server_id': '97d6538b-2585-40fc-b68c-e9a088724a39', 'body': {'volumeAttachment': {'volumeId': 'b8661986-db4e-4719-956e-d387648adac9'}}} Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:02:30.165557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 97d6538b-2585-40fc-b68c-e9a088724a39 while it is in vm_state rescued Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:02:30.166597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 97d6538b-2585-40fc-b68c-e9a088724a39 while it is in vm_state rescued Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Returning 409 to user: Cannot 'attach_volume' instance 97d6538b-2585-40fc-b68c-e9a088724a39 while it is in vm_state rescued {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c2780637-c1dd-48a2-a434-10554915ce58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.039739 Jul 24 01:02:30.167739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 118/228] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:02:30 2022] POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/os-volume_attachments => generated 152 bytes in 41 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Jul 24 01:02:30.173888 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:30.174509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:30.180747 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.181199 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.181967 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.182401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.247695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-db262481-fd05-4d30-9e97-845fbd7d3d40 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/action" status: 202 len: 0 microversion: 2.1 time: 0.077474 Jul 24 01:02:30.248247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 111/229] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:02:30 2022] POST /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39/action => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:02:30.259622 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:30.260500 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:30.272107 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.273271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.274293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.274966 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.432462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:30.441543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9b01e81b-a6c3-4b7d-980d-c8d210348038 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.187396 Jul 24 01:02:30.448234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 119/230] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:30 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:30.490157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3abfea65-7ffe-41de-a680-bd5a8ed5ccdc"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:30.491425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:30.497556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.498076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:30.498818 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:30.499254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:31.037788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f6236534-68c5-414c-aee7-1ffc50c7aca1 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.552736 Jul 24 01:02:31.039229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 112/231] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:02:30 2022] POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-volume_attachments => generated 194 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:31.460414 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:31.461210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:31.467069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:31.467518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:31.468306 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:31.468740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:31.602579 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:31.610473 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-886ac4a4-d23f-4c28-aa97-f5eb8a847447 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.154124 Jul 24 01:02:31.612270 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 120/232] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:31 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:32.631212 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:32.631913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:32.638341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:32.638812 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:32.639588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:32.640024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:32.786045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:32.793702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-86d7745d-4f03-4829-9de5-906e32be0387 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.167420 Jul 24 01:02:32.794520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 113/233] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:32 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:33.811827 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:33.812445 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:33.819675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:33.820127 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:33.820915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:33.821352 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:33.963872 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:33.975715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ca8f4cfe-8b34-40de-89d9-b2d8a65d844b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.167711 Jul 24 01:02:33.976528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 121/234] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:33 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:34.996649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:34.997283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:35.004496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:35.004940 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:35.005694 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:35.006121 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:35.132786 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:35.140275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-cd35c419-1fc9-4f34-ad8a-d0beb880294e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.147507 Jul 24 01:02:35.140839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 114/235] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:34 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:36.157257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:36.159165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:36.161095 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=36 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:36.166312 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:36.166783 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:36.167566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:36.168006 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:36.318562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:36.334878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9bf248ab-0615-443e-8296-65a193fc02dd tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.181193 Jul 24 01:02:36.335860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 122/236] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:36 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:37.357397 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:37.357924 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:37.360309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=30 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:37.366349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:37.366809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:37.367571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:37.368000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:37.504821 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:37.512333 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-92045d48-5692-4e35-b2e5-b5a13ab4feb3 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.158867 Jul 24 01:02:37.512900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 115/237] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:37 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:38.529265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:38.529871 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:38.536555 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:38.537003 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:38.537766 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:38.538201 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:38.666456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:38.673980 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6218dbdb-d9f3-42fb-9a18-f8606fb3e669 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1508 microversion: 2.1 time: 0.148447 Jul 24 01:02:38.674655 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 123/238] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:38 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1508 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:39.065704 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:39.066223 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:39.073111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:39.073564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:39.074339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:39.074795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:39.098636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0ab12e1e-9238-4104-93c7-8a22829b323a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/ips" status: 200 len: 151 microversion: 2.20 time: 0.037367 Jul 24 01:02:39.099283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 116/239] 10.222.0.18 () {60 vars in 1288 bytes} [Sun Jul 24 01:02:39 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/ips => generated 151 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:02:39.691930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:39.692570 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:39.698822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:39.699278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:39.700043 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:39.700482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:39.834571 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:39.842027 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8ddb02d4-89be-48cd-9f71-20bf071225be tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1514 microversion: 2.1 time: 0.154708 Jul 24 01:02:39.842584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 124/240] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:39 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:41.195690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b16ad271-ebc8-4128-b7bb-9f76685a298b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'create', calling method: .version_select at 0x7f5e30686310>, body: {"keypair": {"name": "tempest-keypair-1290565022", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGHlwGWyYyK26YkE/e25M/AAUWoc/Jv/o7QZdYw2iD1kKVC8lq5p1tucJ1qz3S9laYR9pliZDew7l9Z9LIxN4YuGa6vXcMRY9JcplSbCsz4DWrzU9hb3mgKXTRxPuDX6kQ=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:41.209787 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-b16ad271-ebc8-4128-b7bb-9f76685a298b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:02:41.213655 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-b16ad271-ebc8-4128-b7bb-9f76685a298b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for user 283a1bfe0e3b45aaa0a33e5fe6b1d85a and project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:02:41.237289 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b16ad271-ebc8-4128-b7bb-9f76685a298b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047740 Jul 24 01:02:41.237899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 117/241] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:02:41 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:42.056425 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:42.057706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:42.063733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:42.064501 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:42.065295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:42.065763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:42.093408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lazy-loading 'flavor' on Instance uuid 281e8695-c8e7-4293-8abe-a4f88f175084 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:02:42.120584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=100 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:42.399434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5fbd895d-e224-4950-a566-e1bf4fe7f392 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/action" status: 202 len: 0 microversion: 2.20 time: 0.347632 Jul 24 01:02:42.400083 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 125/242] 10.222.0.18 () {62 vars in 1318 bytes} [Sun Jul 24 01:02:42 2022] POST /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/action => generated 0 bytes in 349 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:02:42.417988 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:42.420933 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:42.428327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:42.429001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:42.430181 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:42.430791 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:42.626000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:42.635395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:42.637079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:42.638365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-77e36efc-1983-4edd-bbe9-e728b5d18f4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.233883 Jul 24 01:02:42.639195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 118/243] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:42 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:02:42.985240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-991834683", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd9f01d9-09a8-410d-9e7d-efac033dcd4b"}], "security_groups": [{"name": "tempest-securitygroup--61159647"}], "key_name": "tempest-keypair-1290565022", "user_data": "}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:43.091891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] validate_networks() for [('fd9f01d9-09a8-410d-9e7d-efac033dcd4b', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:02:43.574636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:02:43.576102 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:02:43.576692 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:02:43.577287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:02:43.581033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:02:43.592041 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:43.592487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:43.593293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:43.593731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:43.612617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:02:43.616736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Getting quotas for user 283a1bfe0e3b45aaa0a33e5fe6b1d85a and project f7d219e82efe44f9b6f860a5fabc7733. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:02:43.628975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:02:43.631374 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:02:43.657560 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:43.658262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:43.664875 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:43.665328 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:43.666104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:43.666606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:43.708409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6afca993-26e5-4c21-b941-c6a69470c582 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 0.727969 Jul 24 01:02:43.709337 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 126/244] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:02:42 2022] POST /compute/v2.1/servers => generated 407 bytes in 730 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:02:43.719484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ebfb487e-21a6-4776-8cec-9fd2e00e88c0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:43.720452 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ebfb487e-21a6-4776-8cec-9fd2e00e88c0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:43.850655 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ebfb487e-21a6-4776-8cec-9fd2e00e88c0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:43.875696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ebfb487e-21a6-4776-8cec-9fd2e00e88c0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1245 microversion: 2.1 time: 0.160077 Jul 24 01:02:43.882933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 127/245] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:43 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1245 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:43.909528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:43.918098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:43.919704 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:43.920863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d94f3a28-7346-415a-a732-8e1ae1354a9f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.267229 Jul 24 01:02:43.921420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 119/246] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:43 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:44.895561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:44.896165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:44.903293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:44.903750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:44.904518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:44.905236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:44.944578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:44.945253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:44.952379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:44.952827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:44.953596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:44.954039 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:44.987682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:44.995229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f30eab54-9972-40e2-b11b-ea5a9309712e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1241 microversion: 2.1 time: 0.104711 Jul 24 01:02:44.995809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 128/247] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:44 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1241 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:45.092776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:45.103217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:45.104919 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:45.106077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d6aead63-b218-460c-85f9-ab0cabed367d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.166939 Jul 24 01:02:45.106636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 120/248] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:44 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:02:46.013883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:46.015078 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:46.022182 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:46.022677 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:46.023498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:46.023934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:46.125485 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:46.126193 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:46.133233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:46.133917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:46.134823 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:46.135437 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:46.243905 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:46.251485 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8f1956f1-7117-48cb-94f5-0069164ee355 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1241 microversion: 2.1 time: 0.242200 Jul 24 01:02:46.252052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 129/249] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:46 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1241 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:46.304294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:46.312046 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:46.313548 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:46.314697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-31add6e7-f949-4aae-9841-c0bee2c1ea3b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.193450 Jul 24 01:02:46.315356 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 121/250] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:46 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:47.147243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e636d0fd-6652-4979-9f27-4b406741e148 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:47.156295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e636d0fd-6652-4979-9f27-4b406741e148 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.156937 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e636d0fd-6652-4979-9f27-4b406741e148 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.245838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e636d0fd-6652-4979-9f27-4b406741e148 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Going to try to stop instance {{(pid=117955) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 24 01:02:47.268645 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:47.269801 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:47.276195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.276638 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.277729 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.278244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.299258 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e636d0fd-6652-4979-9f27-4b406741e148 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action" status: 202 len: 0 microversion: 2.1 time: 0.155928 Jul 24 01:02:47.300442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 130/251] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:02:47 2022] POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action => generated 0 bytes in 158 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:02:47.321691 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:47.322380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:47.333239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.333837 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.334619 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.335085 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.353613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:47.360979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0c563581-2c17-48d6-bfcd-e08a7d16f76d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1303 microversion: 2.1 time: 0.096308 Jul 24 01:02:47.361539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 122/252] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:47 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1303 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:47.371616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:47.372339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:47.379309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.379756 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.380532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:47.380969 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:47.532875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:47.543685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6a03704e-bcef-4737-9924-c5991266e2d4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.230095 Jul 24 01:02:47.548593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 131/253] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:47 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:47.560081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:47.570044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:47.572005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:47.573175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6a57cbab-76bd-4f3c-af9c-26349800f81d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.205410 Jul 24 01:02:47.573746 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 123/254] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:47 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:02:48.290069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:48.290610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:48.296932 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.297378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.298147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.298572 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.379479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:48.380218 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:48.400737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.402044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.404797 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.406639 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.462132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lazy-loading 'services' on Instance uuid e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:02:48.571501 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:48.574150 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:48.580591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b184e91a-e307-4849-84ca-ab59d1c6e77f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.206437 Jul 24 01:02:48.581342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 124/255] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:48 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:48.585760 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-653e3237-f94d-401a-8668-ce9e16b96d2d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "DELETE /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696/os-volume_attachments/8ef10991-dc64-4e53-aab2-c929ece8ecbb" status: 202 len: 0 microversion: 2.1 time: 0.299417 Jul 24 01:02:48.587378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 132/256] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:02:48 2022] DELETE /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696/os-volume_attachments/8ef10991-dc64-4e53-aab2-c929ece8ecbb => generated 0 bytes in 302 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:02:48.593033 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:48.593894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:48.600475 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.601235 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.602836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.603635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.615903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:48.617269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:48.630690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.631254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.632077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.632520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.844891 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:48.853741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-da8af39f-65bc-4ab7-90a6-e255dc1f1f0e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.270955 Jul 24 01:02:48.854319 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 125/257] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:48 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:48.874156 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:48.883508 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:48.885292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:48.886651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c7f3bd6b-1fda-43b1-99a2-43a186f36709 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.279796 Jul 24 01:02:48.887242 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 133/258] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:48 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:02:48.962992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-3e580e47-484d-419f-b633-ff3617ea555b req-18e57f61-2e2e-4a74-be84-916db13ad0b6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6a71fa6-32c2-446f-b9e5-80d4e9ea800e", "tag": "c3604b8e-6ace-40d2-8c7a-9f222b26d78c"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:48.974309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3e580e47-484d-419f-b633-ff3617ea555b req-18e57f61-2e2e-4a74-be84-916db13ad0b6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:48.974760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3e580e47-484d-419f-b633-ff3617ea555b req-18e57f61-2e2e-4a74-be84-916db13ad0b6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:48.986984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-3e580e47-484d-419f-b633-ff3617ea555b req-18e57f61-2e2e-4a74-be84-916db13ad0b6 service nova] Creating event network-changed:c3604b8e-6ace-40d2-8c7a-9f222b26d78c for instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e on hv-cinder-maste Jul 24 01:02:48.992181 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-3e580e47-484d-419f-b633-ff3617ea555b req-18e57f61-2e2e-4a74-be84-916db13ad0b6 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032981 Jul 24 01:02:48.993144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 126/259] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:02:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:49.598239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:49.598878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:49.604931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:49.605388 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:49.606206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:49.606643 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:49.750056 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:49.757944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1e70fc99-a3d7-4a21-b288-9802d994e601 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.164144 Jul 24 01:02:49.758519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 134/260] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:49 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:49.872235 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:49.872943 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:49.879365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:49.879817 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:49.880598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:49.881058 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:49.905580 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:49.906191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:49.913172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:49.913947 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:49.914928 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:49.915547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:50.072274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:50.080276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-248eb30b-5a7e-452e-aa95-84ee31755929 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.212852 Jul 24 01:02:50.080846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 127/261] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:49 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:50.098882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:50.106787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:50.108310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:50.109484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d68ec882-9222-4823-aef9-21bd51bf9805 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.207963 Jul 24 01:02:50.110030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 135/262] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:49 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:50.774560 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:50.775267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:50.781501 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:50.781974 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:50.782810 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:50.783246 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:50.810000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:50.810585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:50.817202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:50.817644 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:50.818399 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:50.818850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:50.843605 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:02:50.916294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:50.924366 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-31c18fed-3af1-4f22-98a7-89f0c75a1b8e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.153537 Jul 24 01:02:50.925559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 128/263] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:50 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:50.927312 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6633810b-06e1-4669-8da3-994d194f8223 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "DELETE /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 204 len: 0 microversion: 2.1 time: 0.120761 Jul 24 01:02:50.927889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 136/264] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:02:50 2022] DELETE /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:02:50.935669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:50.936272 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:50.942342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:50.942812 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:50.943570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:50.944265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:51.076493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:51.084192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3ca5f290-3ab8-47a6-aa2b-334352a69d9d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1512 microversion: 2.1 time: 0.152228 Jul 24 01:02:51.084762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 129/265] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:50 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1512 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:51.098824 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:51.099567 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:51.105650 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:51.106099 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:51.106903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:51.107341 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:51.127600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:51.128258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:51.134207 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:51.134658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:51.135469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:51.135902 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:51.220058 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=4 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:51.304437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:51.308766 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:51.312497 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-af178560-2771-43c6-9753-1a7301c94730 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.218445 Jul 24 01:02:51.313077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 137/266] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:51 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:51.318216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:51.320093 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:51.321256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f0b531bf-d2db-43ed-b0af-cf1f4f34352b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.197625 Jul 24 01:02:51.321831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 130/267] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:51 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:51.942006 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:51.942610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:51.948942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:51.949392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:51.950155 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:51.950588 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.083446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:52.090906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-447a60fb-a066-4b2b-87e9-db877df4d7bf tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.153376 Jul 24 01:02:52.091468 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 138/268] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:51 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:52.101839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:52.102476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:52.109298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:52.109739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.110522 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:52.110987 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.239156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:52.246561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ee5c801e-ce4e-400f-a991-5f01c95631a2 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1512 microversion: 2.1 time: 0.148576 Jul 24 01:02:52.247152 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 131/269] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:52 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1512 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:52.329741 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:52.330347 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:52.336675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:52.337126 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.337889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:52.338327 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.339471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:52.340074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:52.341066 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:UPDATE=4,nova_cell1:SELECT=92,nova_cell1:INSERT=4 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:52.346932 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:52.347380 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.348149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:52.348583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:52.539222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:52.542069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:52.547200 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d3b393e3-5db2-4c32-aef9-06893b32c103 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.221236 Jul 24 01:02:52.547949 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 139/270] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:52 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:52.551429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:52.552962 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:52.554121 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-abe39413-dc66-43f0-b19e-302947e0084f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.218605 Jul 24 01:02:52.554755 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 132/271] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:52 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:53.107614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:53.108227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:53.114339 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.115104 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.115864 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.116300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.247764 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:53.255607 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1bbd8f85-754a-4b3f-93ac-c6c8e3040850 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.151839 Jul 24 01:02:53.256179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 140/272] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:53 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:53.264488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:53.265070 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:53.271857 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.272298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.273061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.273494 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.400266 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:53.408174 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-53b5bab2-8588-44a2-838c-f18dfb31eadb tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1512 microversion: 2.1 time: 0.147539 Jul 24 01:02:53.409001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 133/273] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:53 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1512 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:53.564556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:53.565147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:53.572257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.572702 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.572948 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:53.573456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.573554 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:53.574241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.579507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.579955 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.580725 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:53.581313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:53.606178 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:53.774051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:53.777031 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:53.781990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bf7c1860-7c63-4f8b-bb43-94fdfaa46c68 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.221227 Jul 24 01:02:53.782564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 141/274] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:53 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:53.785027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:53.786546 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:53.787732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b9104c76-ee83-4279-b966-a7666e3c6772 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.219012 Jul 24 01:02:53.788280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 134/275] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:53 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:54.277413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:54.278190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:54.280076 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=8 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:54.285243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.285698 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.286503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.286974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.420548 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:54.425891 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:54.427757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:54.430416 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-04cdb4f1-fa39-4da7-990d-4b619a79f4ca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.156310 Jul 24 01:02:54.436685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.437464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.438423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.438885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.440938 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 142/276] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:54 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:54.582471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:54.590589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d53ead99-0be7-4d92-9b4b-71c1d0e12c5a tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1512 microversion: 2.1 time: 0.169280 Jul 24 01:02:54.591296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 135/277] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:54 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1512 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:02:54.799463 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:54.801060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:54.808917 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.809839 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.811972 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.813072 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.814185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:54.814916 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:54.823865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.824236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:54.825146 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:54.825954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.052278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:55.057156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:55.064565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-213d4d7f-51e0-463f-b708-c12bbbfae2ec tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.266469 Jul 24 01:02:55.066536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 143/278] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:54 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:55.076742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:55.077369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-991e556c-7077-47ab-acde-6b619b6c2224 req-f057ed76-00d3-4369-8cb7-0ad619ce992d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6e7b2bf-8a97-487f-ae6d-fdfedd53b696", "name": "network-vif-deleted", "tag": "7d0e4b10-307a-45ff-8fcf-b60161deca8f"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:55.078741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:55.080305 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-22d7d646-a397-4539-858a-763e6073311c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.277738 Jul 24 01:02:55.082620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 136/279] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:54 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:55.099830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-991e556c-7077-47ab-acde-6b619b6c2224 req-f057ed76-00d3-4369-8cb7-0ad619ce992d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:55.100600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-991e556c-7077-47ab-acde-6b619b6c2224 req-f057ed76-00d3-4369-8cb7-0ad619ce992d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.116575 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-991e556c-7077-47ab-acde-6b619b6c2224 req-f057ed76-00d3-4369-8cb7-0ad619ce992d service nova] Creating event network-vif-deleted:7d0e4b10-307a-45ff-8fcf-b60161deca8f for instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 on hv-cinder-maste Jul 24 01:02:55.132625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-991e556c-7077-47ab-acde-6b619b6c2224 req-f057ed76-00d3-4369-8cb7-0ad619ce992d service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064406 Jul 24 01:02:55.133439 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 144/280] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:02:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:55.446625 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:55.447381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:55.454254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:55.454724 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.455616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:55.456123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.582456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:55.589797 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6db9af1a-4d7d-4096-b196-55fabb09ef67 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.147284 Jul 24 01:02:55.590352 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 137/281] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:55 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:55.607744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:55.608438 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:55.615397 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:55.615843 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.616629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:55.617092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.646960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:55.647407 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:55.647852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lazy-loading 'fault' on Instance uuid e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:02:55.695916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:55.703727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ce7d2fe1-89e2-41fd-8287-b7a5ea560888 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 200 len: 1309 microversion: 2.1 time: 0.100176 Jul 24 01:02:55.704297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 145/282] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:55 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 1309 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:56.083324 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:56.083845 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:56.090044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.090489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.091297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.091723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.099657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:56.100264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:56.106207 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.106655 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.107452 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.107888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.293214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:56.293410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:56.304433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:56.305340 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-560b8e94-6571-4710-a7c0-b5c219ddc041 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.226964 Jul 24 01:02:56.306138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:56.306576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 138/283] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:56 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:56.317236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-52532f34-fa2e-436c-86be-8573ba23c556 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.222036 Jul 24 01:02:56.319874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 146/284] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:56 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:56.447375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-66e59efa-cf22-4cb0-8198-393776a56ccc req-97922345-e0b5-4af2-87ad-d0d9f9c27b3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c6a71fa6-32c2-446f-b9e5-80d4e9ea800e", "name": "network-vif-plugged", "status": "completed", "tag": "c3604b8e-6ace-40d2-8c7a-9f222b26d78c"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:02:56.456778 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-66e59efa-cf22-4cb0-8198-393776a56ccc req-97922345-e0b5-4af2-87ad-d0d9f9c27b3e service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.457224 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-66e59efa-cf22-4cb0-8198-393776a56ccc req-97922345-e0b5-4af2-87ad-d0d9f9c27b3e service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.468507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-66e59efa-cf22-4cb0-8198-393776a56ccc req-97922345-e0b5-4af2-87ad-d0d9f9c27b3e service nova] Creating event network-vif-plugged:c3604b8e-6ace-40d2-8c7a-9f222b26d78c for instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e on hv-cinder-maste Jul 24 01:02:56.473497 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-66e59efa-cf22-4cb0-8198-393776a56ccc req-97922345-e0b5-4af2-87ad-d0d9f9c27b3e service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030158 Jul 24 01:02:56.474270 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 139/285] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:02:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:02:56.607366 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:56.608257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:56.616413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.616858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.617630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.618061 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.721183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:56.722171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: e6e7b2bf-8a97-487f-ae6d-fdfedd53b696] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:56.728624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.729086 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.729926 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:56.730661 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:56.745468 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] HTTP exception thrown: Instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 could not be found. Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Request method failure captured: Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 HTTP/1.1 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnjXx2dOBLlVMg0fmFkzZItJuSIa81Gw3chUF7SqSm6RRmy6iQSnm3Ld0Id92t0S_UNzg_lFrC2CEk77o7zqqCltZy9KUcVcwZh10Fms4RgrniL5S8EPtPeFtDlO7tsT3QTj_DMOEdZ3MNeswWkakQBNUrVUjeNGHQUrvfXkeOLFV0 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 4dd26bb2797345d6b531f5ba5db5cbc8 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-VolumesSnapshotTestJSON-425251886 Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:02:56.747822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/4dd26bb2797345d6b531f5ba5db5cbc8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/4dd26bb2797345d6b531f5ba5db5cbc8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/4dd26bb2797345d6b531f5ba5db5cbc8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_4dd26bb2797345d6b531f5ba5db5cbc8", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-VolumesSnapshotTestJSON-425251886 Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 4dd26bb2797345d6b531f5ba5db5cbc8 Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-425251886 Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-VolumesSnapshotTestJSON-425251886-project Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 0e4b9bb864f24c93b2c9547432b8fd96 Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-VolumesSnapshotTestJSON-425251886-project Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 could not be found. Jul 24 01:02:56.749157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'e6e7b2bf-8a97-487f-ae6d-fdfedd53b696'} Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 could not be found. Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:02:56.749914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 could not be found. Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 could not be found. Jul 24 01:02:56.752065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:02:56.753744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Returning 404 to user: Instance e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:02:56.753744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-688e01b9-26bf-4b15-91d1-f0cb45a9029b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696" status: 404 len: 111 microversion: 2.1 time: 0.032532 Jul 24 01:02:56.753744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 140/286] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:56 2022] GET /compute/v2.1/servers/e6e7b2bf-8a97-487f-ae6d-fdfedd53b696 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:02:56.766302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:56.775055 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-529407ff-120e-4fb8-b8c8-11c9b474d740 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.171519 Jul 24 01:02:56.783257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 147/287] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:56 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:57.322290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:57.322931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:57.329095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:57.329559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:57.330359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:57.331362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:57.336356 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:57.336961 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:57.342959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:57.343408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:57.344171 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:57.344602 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:57.536126 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:57.538855 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:57.544339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e5af9e36-fa8a-4f54-a6a9-9d8d3e77c266 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.226180 Jul 24 01:02:57.544913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 141/288] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:57 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:57.546852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:57.548353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:57.549547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-97799687-e9db-4db3-9f07-80d684c35729 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.217062 Jul 24 01:02:57.550283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 148/289] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:57 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:57.792014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:57.792631 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:57.798914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:57.799362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:57.800143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:57.800571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:57.926299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:57.934192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-abcfc771-05ab-4261-b015-cdce87a44b9c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.146100 Jul 24 01:02:57.934760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 142/290] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:57 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:58.561387 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:58.562147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:58.568435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:58.569299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:58.569941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:58.570464 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:58.571419 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:58.571928 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:58.575691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:58.576138 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:58.576897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:58.577323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:58.592633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=100 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:02:58.766509 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:58.771883 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:58.775600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-79f92c5a-c50c-4147-a98c-682086f17ee9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.218012 Jul 24 01:02:58.776167 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 149/291] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:58 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:58.779977 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:58.781504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:58.782701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e963fb44-8b9c-40e7-a645-81147bd88f2d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.218808 Jul 24 01:02:58.783285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 143/292] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:58 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:02:58.951960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:58.952500 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:58.958665 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:58.960626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:58.961415 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:58.961862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:59.092978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:02:59.101349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f96b9140-4eac-441c-a827-bb07cacd099a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.153138 Jul 24 01:02:59.101922 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 150/293] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:58 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:02:59.792573 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:59.793183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:59.800611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:59.800817 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:59.801541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:02:59.801931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:59.802020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:59.802144 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:02:59.808222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:59.808668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:02:59.809426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:02:59.809858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:00.002601 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:00.005654 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:00.013990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:00.015522 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:00.016277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1a459030-62c5-4950-8af2-151b01b56c42 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.227490 Jul 24 01:03:00.016695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a10beb12-4432-4245-9cd9-c8bc5eedd87c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.219778 Jul 24 01:03:00.017024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 144/294] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:02:59 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:00.017238 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 151/295] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:02:59 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:00.119342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:00.119928 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:00.125887 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:00.126337 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:00.127142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:00.127586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:00.257003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:00.264445 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e8972436-55dd-4672-bf70-7883ab3b272f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1369 microversion: 2.1 time: 0.149024 Jul 24 01:03:00.265014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 145/296] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:00 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1369 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:01.034822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:01.035451 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:01.036067 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:01.036687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:01.047236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.047236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.047411 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.047490 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.047562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.047632 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.047711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.047711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.269313 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:01.284765 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:01.286882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:01.291696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:01.292976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-07890ec6-b371-475d-8d3e-2b65d9e8c469 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.261946 Jul 24 01:03:01.293550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 146/297] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:01 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:01.294555 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-08c94d9c-eab3-43a2-ab26-f6802b8ce4fd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.263519 Jul 24 01:03:01.295784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 152/298] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:01 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:01.298736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:01.300045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:01.307262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.307704 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.308471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.308902 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.436583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:01.444712 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-757f8a1b-0f77-4b68-a23a-3c883aa99fca tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1562 microversion: 2.1 time: 0.149785 Jul 24 01:03:01.445271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 147/299] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:01 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1562 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:01.461356 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:01.461852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:01.468067 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.468562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.469339 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:01.469798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:01.546745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-38b46298-f24c-460b-87c2-c1cc3f27fc7a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089267 Jul 24 01:03:01.547375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 153/300] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:03:01 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:02.311329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:02.312517 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:02.316979 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:02.318094 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:02.325838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:02.326553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:02.326679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:02.327044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:02.327844 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:02.328282 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:02.328509 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:02.329083 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:02.662394 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:02.670624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:02.672177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:02.673332 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-904c45f7-1ae5-4ce8-b8af-5e810313bb19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1743 microversion: 2.20 time: 0.363847 Jul 24 01:03:02.673879 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 154/301] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:02 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1743 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:02.706240 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:02.716633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-633a4c2b-e48a-450a-b22e-722e19622940 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.410242 Jul 24 01:03:02.717628 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 148/302] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:02 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:02.942947 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-04490414-163b-4e11-99bf-08e534b90428 req-65ddd643-41f8-47d4-be85-8bd36e0ca759 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6a71fa6-32c2-446f-b9e5-80d4e9ea800e", "tag": "c3604b8e-6ace-40d2-8c7a-9f222b26d78c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:02.953939 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-04490414-163b-4e11-99bf-08e534b90428 req-65ddd643-41f8-47d4-be85-8bd36e0ca759 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:02.954385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-04490414-163b-4e11-99bf-08e534b90428 req-65ddd643-41f8-47d4-be85-8bd36e0ca759 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:02.965866 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-04490414-163b-4e11-99bf-08e534b90428 req-65ddd643-41f8-47d4-be85-8bd36e0ca759 service nova] Creating event network-changed:c3604b8e-6ace-40d2-8c7a-9f222b26d78c for instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e on hv-cinder-maste Jul 24 01:03:02.972806 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-04490414-163b-4e11-99bf-08e534b90428 req-65ddd643-41f8-47d4-be85-8bd36e0ca759 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033098 Jul 24 01:03:02.973552 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 155/303] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:03.693500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:03.694490 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:03.701171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:03.701616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:03.702588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:03.702915 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:03.735929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:03.737445 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:03.744372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:03.745002 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:03.745971 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:03.746639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:03.903561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:03.913399 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:03.915271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:03.916625 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-53f8eba7-8ea5-4096-b4ba-24f3ebb7f6df tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1743 microversion: 2.20 time: 0.227790 Jul 24 01:03:03.925707 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 149/304] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:03 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1743 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:03.928732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:03.937307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2b80c9e5-496d-4c37-8611-d21967570281 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.205391 Jul 24 01:03:03.937902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 156/305] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:03 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:04.526174 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:04.526598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:04.532731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:04.533190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:04.534182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:04.534644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:04.556950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-35492143-5b48-4c9c-b4ce-e062d7166bb7 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/ips" status: 200 len: 150 microversion: 2.1 time: 0.034769 Jul 24 01:03:04.557512 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 150/306] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:03:04 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/ips => generated 150 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:04.935706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:04.936405 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:04.943535 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:04.944249 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:04.945028 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:04.945474 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:04.948209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=100 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:04.955877 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:04.956579 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:04.963088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:04.963543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:04.964313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:04.964752 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:05.144077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:05.150212 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:05.152310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:05.153810 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:05.154999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b8bc038f-0749-4926-b5d5-822925985a37 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1743 microversion: 2.20 time: 0.223299 Jul 24 01:03:05.155636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 157/307] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:04 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1743 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:05.158174 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-18b33575-f67f-472a-9003-430dfb70ee17 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.206142 Jul 24 01:03:05.159745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 151/308] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:04 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:06.174870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:06.175710 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:06.176105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:06.176330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:06.177905 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:06.183168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:06.183356 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:06.183622 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:06.183834 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:06.184398 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:06.184723 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:06.184838 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:06.185259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:06.388585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:06.395684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:06.397773 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:06.399565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:06.400991 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a0f9a174-60fc-4e6a-83f3-ad3e51e8da2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1743 microversion: 2.20 time: 0.230151 Jul 24 01:03:06.401551 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 158/309] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:06 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1743 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:06.404261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1d9db3bf-ecb3-4098-be92-46421f771a9d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.232577 Jul 24 01:03:06.405077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 152/310] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:06 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:06.777182 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1642456235", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "networks": [{"uuid": "817ac8a4-2b25-4742-bdf4-9d6cc008d45a"}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:06.830307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] validate_networks() for [('817ac8a4-2b25-4742-bdf4-9d6cc008d45a', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:03:06.979840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "af294a16-ac68-4fe3-b3fb-db77a673a732"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:06.981088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:06.988382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:06.989174 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:06.990184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:06.990874 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.244871 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:03:07.245376 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:03:07.246034 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:03:07.246516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:03:07.247015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:03:07.259589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:07.260043 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.260861 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:07.261477 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.280579 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Getting quotas for project 4dd26bb2797345d6b531f5ba5db5cbc8. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:03:07.285039 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Getting quotas for user 0e4b9bb864f24c93b2c9547432b8fd96 and project 4dd26bb2797345d6b531f5ba5db5cbc8. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:03:07.299912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:03:07.302706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:03:07.389921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fc22a243-96af-4067-9aa2-5852e98def59 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.616400 Jul 24 01:03:07.391391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 159/311] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:03:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 618 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:03:07.406195 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-82e68806-3e61-43e9-aa4c-2572df8cf86f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:07.407627 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-82e68806-3e61-43e9-aa4c-2572df8cf86f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:07.498350 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f939019d-051d-4dbe-81ff-85678ed07107 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.528218 Jul 24 01:03:07.499012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 153/312] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:03:06 2022] POST /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments => generated 194 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:07.499261 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-82e68806-3e61-43e9-aa4c-2572df8cf86f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:07.507425 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-82e68806-3e61-43e9-aa4c-2572df8cf86f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1219 microversion: 2.1 time: 0.109536 Jul 24 01:03:07.510864 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 160/313] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:07 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1219 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:07.521642 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:07.523761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:07.523983 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:07.524586 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:07.528699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:07.543012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:07.548792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:07.549835 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.550744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:07.550934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.551268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.552276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:07.552901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:07.821753 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:07.832246 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:07.836525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:07.838929 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8128aab0-8c48-49c6-9605-5598312e572c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.329479 Jul 24 01:03:07.840168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 154/314] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:07 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:07.840744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:07.850362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-51ec7379-e591-409e-832f-a8c69c68e834 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.338272 Jul 24 01:03:07.866076 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 161/315] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:07 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:08.507742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-7769ce8c-d06b-4e2e-91b6-375875829216 req-e5937cad-3da3-4592-9bd8-afe87735c7fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "name": "network-vif-plugged", "status": "completed", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:08.520133 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-7769ce8c-d06b-4e2e-91b6-375875829216 req-e5937cad-3da3-4592-9bd8-afe87735c7fe service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.527916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:08.529361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:08.529723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-7769ce8c-d06b-4e2e-91b6-375875829216 req-e5937cad-3da3-4592-9bd8-afe87735c7fe service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:08.537049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.538492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:08.539639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.540448 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:08.550338 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-7769ce8c-d06b-4e2e-91b6-375875829216 req-e5937cad-3da3-4592-9bd8-afe87735c7fe service nova] Creating event network-vif-plugged:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:03:08.556149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-7769ce8c-d06b-4e2e-91b6-375875829216 req-e5937cad-3da3-4592-9bd8-afe87735c7fe service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053868 Jul 24 01:03:08.557463 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 155/316] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:08.623875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:08.638444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bf6c22dd-a4b8-4824-810e-136ba97a2c76 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1271 microversion: 2.1 time: 0.115126 Jul 24 01:03:08.640401 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 162/317] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:08 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1271 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:08.874613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:08.882746 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:08.898634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.899454 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:08.899454 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:08.903147 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:08.903994 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.907775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.908239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:08.909592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:08.910032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:08.912205 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:09.136406 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:09.147040 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:09.148686 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6f3657ef-4c31-4f13-942e-2b162a1948a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1724 microversion: 2.1 time: 0.279015 Jul 24 01:03:09.149278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 163/318] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:08 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1724 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:09.156613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:09.159467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:09.161169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b64c4796-b3d3-42ca-a8a5-5ac7dcd9600e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.305788 Jul 24 01:03:09.162291 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 156/319] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:08 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:09.662482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:09.664841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:09.680839 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:09.681913 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:09.683007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:09.683668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:09.774279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:09.810762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-98cda4cf-f39a-4d97-a6ed-91865fb68b5e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1277 microversion: 2.1 time: 0.156717 Jul 24 01:03:09.811398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 164/320] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:09 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1277 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:10.169726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:10.171992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:10.182064 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:10.182954 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:10.189345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.193403 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.194892 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.196172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.196910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.198848 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.202061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.205931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.501469 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:10.509839 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:10.511404 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:10.512586 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-243854e3-270f-4678-abd1-846204a701c2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.335356 Jul 24 01:03:10.513308 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:10.513600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 165/321] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:10 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:10.523260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-21202f71-46a9-4504-b3b2-56f062b7516b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1701 microversion: 2.1 time: 0.359232 Jul 24 01:03:10.524090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 157/322] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:10 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1701 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:10.542637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e54f651f-4729-49e6-936f-638966ac2187 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:10.550162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e54f651f-4729-49e6-936f-638966ac2187 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.550721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e54f651f-4729-49e6-936f-638966ac2187 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.599539 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e54f651f-4729-49e6-936f-638966ac2187 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Going to try to start instance {{(pid=117955) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 24 01:03:10.668236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e54f651f-4729-49e6-936f-638966ac2187 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action" status: 202 len: 0 microversion: 2.1 time: 0.128977 Jul 24 01:03:10.670400 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 166/323] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:03:10 2022] POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:03:10.679565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:10.680456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:10.686833 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.687515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.688463 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.689072 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.830931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:10.832087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:10.847992 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.848715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.849973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:10.850576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:10.988155 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:10.999952 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f5af42a2-9705-4f52-addb-03bc85468477 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1710 microversion: 2.1 time: 0.324395 Jul 24 01:03:11.014238 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 158/324] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:10 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1710 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:11.086309 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:11.095289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2fd2da09-7f8f-4dd8-9531-8e38139f4f78 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.269207 Jul 24 01:03:11.096488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 167/325] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:10 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:11.481183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-1f5e2251-3251-4454-a62f-727cb42571db req-b00d6037-4995-425b-8047-0cca9b1802bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c28bad0d-af40-4706-8310-b7212eccb24e", "tag": "260c7eb5-b1ed-40d9-b311-c9db67b1cfaa"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:11.499773 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1f5e2251-3251-4454-a62f-727cb42571db req-b00d6037-4995-425b-8047-0cca9b1802bd service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:11.500271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1f5e2251-3251-4454-a62f-727cb42571db req-b00d6037-4995-425b-8047-0cca9b1802bd service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:11.517026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-1f5e2251-3251-4454-a62f-727cb42571db req-b00d6037-4995-425b-8047-0cca9b1802bd service nova] Creating event network-changed:260c7eb5-b1ed-40d9-b311-c9db67b1cfaa for instance c28bad0d-af40-4706-8310-b7212eccb24e on hv-cinder-maste Jul 24 01:03:11.536415 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:11.537585 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-1f5e2251-3251-4454-a62f-727cb42571db req-b00d6037-4995-425b-8047-0cca9b1802bd service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066546 Jul 24 01:03:11.539080 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:11.540232 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 159/326] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:03:11.552797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:11.553890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:11.555755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:11.556334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:11.794155 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:11.803611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:11.806353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:11.807914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-75740c00-de3f-40e1-92ef-2a547abb11c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.280508 Jul 24 01:03:11.809855 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 168/327] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:11 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:12.024785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:12.025881 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:12.040121 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:12.041136 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:12.049115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:12.050102 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:12.078008 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:12.112842 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:12.113461 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:12.129353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:12.129808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:12.130598 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:12.133678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:12.367070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:12.374693 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d1d733d9-444a-4c09-af92-5c5784ecb116 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.266186 Jul 24 01:03:12.381259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 169/328] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:12 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:12.403523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:12.417973 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5f47c400-e0d7-4ace-bdd0-1ddcf9cdd6b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1710 microversion: 2.1 time: 0.399543 Jul 24 01:03:12.419804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 160/329] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:12 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1710 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:12.826903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:12.828170 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:12.838979 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:12.839704 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:12.843105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:12.845271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:13.020614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:13.030243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:13.032243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:13.035014 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b9342cb6-0174-42c4-b079-764c91214547 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.211690 Jul 24 01:03:13.046806 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 170/330] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:12 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:13.404364 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:13.404968 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:13.422961 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:13.426055 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:13.430615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:13.431645 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:13.436857 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:13.437643 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:13.444801 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:13.445260 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:13.446042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:13.446518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:13.893256 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:13.904770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4d2f2748-3d39-42c6-a311-24c90b52506e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1710 microversion: 2.1 time: 0.472184 Jul 24 01:03:13.909288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 171/331] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:13 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1710 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:13.981866 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:13.991446 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-71e60bd2-07b3-4e96-9b69-357da3d8298d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.593170 Jul 24 01:03:13.992033 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 161/332] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:13 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:14.055117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:14.055854 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:14.062170 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:14.062630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:14.063450 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:14.063892 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:14.215998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:14.231273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:14.233106 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:14.234480 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-de07752c-0b25-498b-b121-0b1cbbc81118 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.183529 Jul 24 01:03:14.246101 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 172/333] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:14 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:14.927144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:14.927759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:14.937122 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:14.937575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:14.938348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:14.938794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:15.010902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:15.013218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:15.023865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:15.024756 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:15.026039 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:15.026822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:15.220332 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:15.230060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9fa0d861-8c65-41a8-b277-8b5ee3f6918e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1714 microversion: 2.1 time: 0.307135 Jul 24 01:03:15.232347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 162/334] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:14 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1714 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:15.253913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:15.255020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:15.261337 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:15.261899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:15.262760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:15.263441 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:15.269648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:15.278024 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c133950a-cb13-491d-9fe8-4e94879df87c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.272347 Jul 24 01:03:15.286803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 173/335] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:15 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:15.428566 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:15.437280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:15.439294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:15.440696 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-38a96309-db3d-4641-986d-cf0db263fa68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.191257 Jul 24 01:03:15.441916 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 163/336] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:15 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:16.294821 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:16.295439 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:16.306800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:16.306800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:16.308267 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:16.308704 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:16.316162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4dd095cf-228e-4728-ba9a-5bd8034da338 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: .version_select at 0x7f5e30686670>, body: {"keypair": {"name": "tempest-keypair-1714377476", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF3ME0tsUUl3FOmvBxdmjjOBoD9uFGpUp8P1bLd/bS1iXvBTy4Mp9Gbfby1fbrO0PneON3Lo7jGUQH5NNGqgKF9pul0qA4O5LQhR3DxT/9xqUBb8aFUYQLxIajPd+kEvfw=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:16.348304 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-4dd095cf-228e-4728-ba9a-5bd8034da338 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:03:16.356788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:16.358301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-4dd095cf-228e-4728-ba9a-5bd8034da338 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:03:16.417927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4dd095cf-228e-4728-ba9a-5bd8034da338 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.106355 Jul 24 01:03:16.425934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 164/337] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:03:16 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:16.465332 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:16.466651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:16.496603 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:16.498514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:16.501685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:16.502179 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:16.542022 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:16.556393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d67fdba8-ada0-45b0-864f-a9ffff5405d7 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.265286 Jul 24 01:03:16.557680 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 174/338] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:16 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:16.777151 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:16.786575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:16.788611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:16.789895 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d6990527-438c-4e33-84be-22941a5c94a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.330491 Jul 24 01:03:16.790717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 165/339] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:16 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:17.573710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:17.575288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:17.590873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:17.592247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=8 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:17.594391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:17.596513 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:17.597749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:17.598567 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:17.808866 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:17.810136 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:17.825596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:17.839545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:17.840115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:17.841022 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:17.841525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:17.844859 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7111d171-3faa-416e-9fcc-7345a33a9f4c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.274936 Jul 24 01:03:17.846564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 175/340] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:17 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:18.277082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:18.285234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:18.286710 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:18.288795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c2c5e401-5b50-4231-a493-b8d164bb6cfb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1738 microversion: 2.20 time: 0.483885 Jul 24 01:03:18.289556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 166/341] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:17 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1738 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:18.492169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1750711409", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "37cd1572-14d0-471e-b2ca-ff3f9078882e"}], "security_groups": [{"name": "tempest-securitygroup--1046328152"}], "key_name": "tempest-keypair-1714377476", "user_data": "}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:18.532816 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-1efd87f4-7151-424a-a5c2-10fb7321460e req-43f947d9-5cec-44f8-beda-89d8e3904a7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c28bad0d-af40-4706-8310-b7212eccb24e", "name": "network-vif-plugged", "status": "completed", "tag": "260c7eb5-b1ed-40d9-b311-c9db67b1cfaa"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:18.543804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1efd87f4-7151-424a-a5c2-10fb7321460e req-43f947d9-5cec-44f8-beda-89d8e3904a7c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:18.544394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1efd87f4-7151-424a-a5c2-10fb7321460e req-43f947d9-5cec-44f8-beda-89d8e3904a7c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:18.558039 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-1efd87f4-7151-424a-a5c2-10fb7321460e req-43f947d9-5cec-44f8-beda-89d8e3904a7c service nova] Creating event network-vif-plugged:260c7eb5-b1ed-40d9-b311-c9db67b1cfaa for instance c28bad0d-af40-4706-8310-b7212eccb24e on hv-cinder-maste Jul 24 01:03:18.563305 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-1efd87f4-7151-424a-a5c2-10fb7321460e req-43f947d9-5cec-44f8-beda-89d8e3904a7c service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034911 Jul 24 01:03:18.564183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 167/342] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:03:18.607048 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] validate_networks() for [('37cd1572-14d0-471e-b2ca-ff3f9078882e', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:03:18.864411 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:18.865117 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:18.871616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:18.872060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:18.873006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:18.873448 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:19.096808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:03:19.096808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:03:19.096808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:03:19.096808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:03:19.097223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:03:19.098098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:19.109030 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e79d95aa-a513-4ebd-a1a8-a75cba30347c tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.249354 Jul 24 01:03:19.113969 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:19.114433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:19.115281 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:19.115725 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:19.116522 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 168/343] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:18 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:19.136716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:03:19.140848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:03:19.153332 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:03:19.156055 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:03:19.214169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-494aa22e-488b-450d-9478-44052c1e0625 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.726033 Jul 24 01:03:19.216157 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 176/344] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:03:18 2022] POST /compute/v2.1/servers => generated 402 bytes in 729 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:03:19.227066 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3b0fdd7d-1683-414c-971a-7e3775db0ba5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:19.227684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3b0fdd7d-1683-414c-971a-7e3775db0ba5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:19.291573 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3b0fdd7d-1683-414c-971a-7e3775db0ba5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:19.298736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3b0fdd7d-1683-414c-971a-7e3775db0ba5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1242 microversion: 2.1 time: 0.076302 Jul 24 01:03:19.309166 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 169/345] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:19 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1242 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:19.310267 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:19.310936 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:19.320307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:19.320779 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:19.321770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:19.322232 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:19.548536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:19.568787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:19.574610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:19.576055 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9c6e81b8-628f-4417-a5e3-bcfed10496be tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1720 microversion: 2.20 time: 0.271297 Jul 24 01:03:19.576625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 177/346] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:19 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1720 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:20.126031 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:20.127284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:20.134893 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:20.135611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:20.136553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:20.137155 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:20.316406 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:20.317639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:20.318603 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:20.325705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:20.326148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:20.327662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a4ad640b-fff7-4a2a-bcae-f21fbc9154c1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1319 microversion: 2.1 time: 0.205522 Jul 24 01:03:20.327953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:20.328570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 170/347] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:20 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1319 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:20.328830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:20.391443 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:20.400499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-194d1d67-13cc-4363-9c0f-5dce81137589 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1238 microversion: 2.1 time: 0.088401 Jul 24 01:03:20.401737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 178/348] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:20 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1238 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:20.595149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:20.596254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:20.602469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:20.603156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:20.604139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:20.604724 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:20.752890 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:20.761504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:20.763042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:20.764200 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ea0cac7f-73f0-4992-a4da-2c2c8ec00c72 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1720 microversion: 2.20 time: 0.173065 Jul 24 01:03:20.764758 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 171/349] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:20 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1720 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:21.346211 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:21.347656 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:21.354111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.355248 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.356225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.356826 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.425703 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:21.426327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:21.438079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.438540 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.442564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.443084 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.531425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:21.541113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fba81715-9557-4b41-93fa-50f12369da2f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1294 microversion: 2.1 time: 0.122214 Jul 24 01:03:21.552459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 172/350] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:21 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1294 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:21.559785 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:21.568615 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-be658bb0-3ca2-48b6-b47d-97a35c8ce3a1 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1506 microversion: 2.1 time: 0.226970 Jul 24 01:03:21.569488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 179/351] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:21 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1506 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:21.586797 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c7517365-72cd-4b65-9480-378e0130c123", "device": "/dev/sdb"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:21.588090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:21.594969 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.596317 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.597318 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.597853 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.783696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:21.784616 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:21.790692 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.791173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.791951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:21.792383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:21.943710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:21.964634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:21.966593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:21.968028 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-78a05f8b-c7f3-4d2c-960c-c639996d3528 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1720 microversion: 2.20 time: 0.189109 Jul 24 01:03:21.968823 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 180/352] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:21 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1720 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:22.559650 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:22.560333 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:22.567177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:22.567631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:22.568706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:22.569695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:22.653592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:22.662227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4fad86a9-20d7-40fe-bcbe-007562d950f6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1294 microversion: 2.1 time: 0.106781 Jul 24 01:03:22.663051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 181/353] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:22 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1294 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:22.728057 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-f343a4a6-79ca-4b6b-b98d-a45a42ec689f req-5cf9b8a5-7add-410d-81c1-bf1129c54c76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "name": "network-vif-unplugged", "status": "completed", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:22.749199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-f343a4a6-79ca-4b6b-b98d-a45a42ec689f req-5cf9b8a5-7add-410d-81c1-bf1129c54c76 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:22.750317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-f343a4a6-79ca-4b6b-b98d-a45a42ec689f req-5cf9b8a5-7add-410d-81c1-bf1129c54c76 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:22.768970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-f343a4a6-79ca-4b6b-b98d-a45a42ec689f req-5cf9b8a5-7add-410d-81c1-bf1129c54c76 service nova] Creating event network-vif-unplugged:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:03:22.775564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-f343a4a6-79ca-4b6b-b98d-a45a42ec689f req-5cf9b8a5-7add-410d-81c1-bf1129c54c76 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.053224 Jul 24 01:03:22.776605 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 182/354] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:22 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:22.990912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:22.993356 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:23.008158 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:23.009408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:23.010369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:23.011005 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:23.226256 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:23.240532 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:23.242363 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:23.243882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a8c48258-c81d-4c39-960d-054d5e779079 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1720 microversion: 2.20 time: 0.258317 Jul 24 01:03:23.244962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 183/355] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:22 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1720 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:23.654493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-86b3d562-5842-440e-acd4-a102524d2f43 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "POST /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.070796 Jul 24 01:03:23.663951 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 173/356] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:03:21 2022] POST /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e/os-volume_attachments => generated 194 bytes in 2082 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:03:23.679599 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:23.680189 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:23.687049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:23.687509 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:23.688631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:23.688728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:23.779514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:23.789934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-69834a19-333f-4732-80ec-0cae383d001a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1312 microversion: 2.1 time: 0.114914 Jul 24 01:03:23.790593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 184/357] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:23 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1312 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:24.266242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:24.271113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:24.278540 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:24.279233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:24.280220 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:24.280697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:24.477784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:24.488440 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:24.491270 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:24.496458 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7738e0f7-3e7b-415a-bfd3-71e150ddab4a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1663 microversion: 2.20 time: 0.236231 Jul 24 01:03:24.497822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 174/358] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:24 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1663 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:24.519241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d78a3415-3fd7-4061-b9c5-cc2f467e75e0"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:24.520437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:24.532139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:24.533481 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:24.534629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:24.535386 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:24.807370 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:24.807966 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:24.821904 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:24.822429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:24.823431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:24.825103 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:24.974061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:24.988020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d1625709-541d-4b15-a6ad-35c002eed034 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1300 microversion: 2.1 time: 0.184364 Jul 24 01:03:24.989391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 175/359] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:24 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1300 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:25.038519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6057bab4-522c-468d-a4b9-ef84b11facf0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.523609 Jul 24 01:03:25.039758 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 185/360] 10.222.0.18 () {62 vars in 1363 bytes} [Sun Jul 24 01:03:24 2022] POST /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments => generated 188 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:03:25.184358 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:25.184921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:25.192352 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:25.193296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:25.194670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:25.195369 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:25.361927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d54f0515-37fa-4de2-ae46-af03deb1187c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/action" status: 202 len: 0 microversion: 2.20 time: 0.180685 Jul 24 01:03:25.367343 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 176/361] 10.222.0.18 () {62 vars in 1318 bytes} [Sun Jul 24 01:03:25 2022] POST /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/action => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:03:25.374238 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:25.375078 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:25.386109 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:25.387205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:25.390921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:25.391442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:25.663212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:25.672765 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:25.675584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:25.676991 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-77fe13e2-ca61-4e71-a402-d93b51a3bc19 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1749 microversion: 2.20 time: 0.308064 Jul 24 01:03:25.681224 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 186/362] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:25 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1749 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:26.007515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:26.008514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:26.020376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:26.021574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:26.022485 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:26.023027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:26.063564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:26.356854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:26.373872 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-aa68ff6c-0331-4685-87be-dab1bdf31c05 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.371241 Jul 24 01:03:26.378233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 177/363] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:26 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:26.474393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-91e7a791-acaf-4617-98e8-6b5b14d5ad32 req-dcb3fe64-d2ef-48ff-83bd-545036a61348 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fd889446-7a8d-4e52-91a5-22b75a923a21", "tag": "0a201010-559e-40e9-8371-e03c2a2e2d4c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:26.493139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-91e7a791-acaf-4617-98e8-6b5b14d5ad32 req-dcb3fe64-d2ef-48ff-83bd-545036a61348 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:26.493583 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-91e7a791-acaf-4617-98e8-6b5b14d5ad32 req-dcb3fe64-d2ef-48ff-83bd-545036a61348 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:26.529897 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-91e7a791-acaf-4617-98e8-6b5b14d5ad32 req-dcb3fe64-d2ef-48ff-83bd-545036a61348 service nova] Creating event network-changed:0a201010-559e-40e9-8371-e03c2a2e2d4c for instance fd889446-7a8d-4e52-91a5-22b75a923a21 on hv-cinder-maste Jul 24 01:03:26.536025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-91e7a791-acaf-4617-98e8-6b5b14d5ad32 req-dcb3fe64-d2ef-48ff-83bd-545036a61348 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065226 Jul 24 01:03:26.536940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 187/364] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:26.697963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:26.699470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:26.706116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:26.710672 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:26.711829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:26.713182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:27.417263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:27.418046 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:27.421242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:27.442033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:27.444120 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:27.446058 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:27.446477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:27.449158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:27.450536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:27.450761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6db0cd78-226b-4883-a0c4-64ba9ff5f81f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1749 microversion: 2.20 time: 0.758059 Jul 24 01:03:27.451602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 178/365] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:26 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1749 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:27.820460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:27.829309 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a967c58b-7b8d-4b7e-9b2b-b8e3a584265e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.438219 Jul 24 01:03:27.830657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 188/366] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:27 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:28.469745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:28.471820 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:28.481498 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:28.482035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:28.482808 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:28.483256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:28.682123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:28.696523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:28.698338 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:28.699756 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e568abb4-2beb-40d9-8fff-94384be2c025 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1749 microversion: 2.20 time: 0.234247 Jul 24 01:03:28.707325 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 179/367] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:28 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1749 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:28.851185 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:28.851816 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:28.875443 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:28.875901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:28.876748 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:28.877237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:29.065437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:29.074883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8864f86b-5f29-46e2-b840-095a01df01bd tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.231442 Jul 24 01:03:29.085152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 189/368] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:28 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:29.725537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:29.726147 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:29.728284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:29.734553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:29.735034 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:29.736008 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:29.736596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:29.888488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:29.899322 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:29.901258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:29.902881 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-bf92db3e-e55d-4561-ac52-d53cd6862517 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1751 microversion: 2.20 time: 0.181227 Jul 24 01:03:29.903693 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 180/369] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:29 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1751 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:30.089169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:30.091311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:30.096867 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:30.097889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:30.105656 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:30.106108 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:30.106906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:30.107501 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:30.124319 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:30.306428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:30.314319 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1d96681b-5727-4d1b-b37d-1ac8cfb8094e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.225739 Jul 24 01:03:30.315081 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 190/370] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:30 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:30.922858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:30.924470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:30.932252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:30.932708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:30.933871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:30.934315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:31.116422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:31.125363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:31.127736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:31.129521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d82d6ed1-39a7-453d-98a9-6a21a417025c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1751 microversion: 2.20 time: 0.210809 Jul 24 01:03:31.130631 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 181/371] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:30 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1751 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:31.334824 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:31.340716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:31.352725 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:31.354298 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:31.355123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:31.355728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:31.588605 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:31.597610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1727f495-36fa-47cf-8c15-14e5d4ec460f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.269172 Jul 24 01:03:31.598207 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 191/372] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:31 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:32.148262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:32.149648 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:32.156958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:32.158124 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:32.159056 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:32.159501 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:32.317242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:32.328847 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:32.330408 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:32.331794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-74864905-ae3c-41f6-9b51-f65495eda536 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1751 microversion: 2.20 time: 0.187724 Jul 24 01:03:32.346184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 182/373] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:32 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1751 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:32.615096 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:32.617792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:32.620940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:32.624551 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:32.624998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:32.625788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:32.626272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:32.813596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:32.824280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-759ffbf2-93b2-4b2b-aef2-b465b7ff3f94 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.212882 Jul 24 01:03:32.834220 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 192/374] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:32 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:33.357088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:33.361284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:33.376795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:33.377637 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:33.379136 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:33.381477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:33.600678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:33.611599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:33.613552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:33.615012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0a07fb6c-1c8d-42db-9232-f52b04769563 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1751 microversion: 2.20 time: 0.265539 Jul 24 01:03:33.616111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 183/375] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:33 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1751 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:33.849860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:33.851396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:33.862286 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:33.862755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:33.863626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:33.864775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:34.127435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:34.146921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8dd776b5-0d77-425a-949d-b6d1b1f2643c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.306411 Jul 24 01:03:34.150791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 193/376] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:33 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:34.634626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:34.635876 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:34.646448 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:34.647157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:34.648216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:34.648846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:34.842168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:34.859103 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:34.871059 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:34.872410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4f14559d-404b-4be6-b504-117fd5227687 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1751 microversion: 2.20 time: 0.242338 Jul 24 01:03:34.873719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 184/377] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:34 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1751 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:35.165826 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:35.166475 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:35.173323 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:35.174117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:35.174948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:35.175394 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:35.409732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:35.426842 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7868c05e-8f73-40df-9530-d96aa0ad9eab tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.264727 Jul 24 01:03:35.428654 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 194/378] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:35 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:35.610322 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-1481db76-44c6-4ab2-b7ee-b4be7c451129 req-b9bc4fc8-e89a-4196-96ec-3ee13fbba8e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd889446-7a8d-4e52-91a5-22b75a923a21", "name": "network-vif-plugged", "status": "completed", "tag": "0a201010-559e-40e9-8371-e03c2a2e2d4c"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:35.624754 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1481db76-44c6-4ab2-b7ee-b4be7c451129 req-b9bc4fc8-e89a-4196-96ec-3ee13fbba8e2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:35.625201 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1481db76-44c6-4ab2-b7ee-b4be7c451129 req-b9bc4fc8-e89a-4196-96ec-3ee13fbba8e2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:35.650000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-1481db76-44c6-4ab2-b7ee-b4be7c451129 req-b9bc4fc8-e89a-4196-96ec-3ee13fbba8e2 service nova] Creating event network-vif-plugged:0a201010-559e-40e9-8371-e03c2a2e2d4c for instance fd889446-7a8d-4e52-91a5-22b75a923a21 on hv-cinder-maste Jul 24 01:03:35.656191 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-1481db76-44c6-4ab2-b7ee-b4be7c451129 req-b9bc4fc8-e89a-4196-96ec-3ee13fbba8e2 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049511 Jul 24 01:03:35.657144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 185/379] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:35.890685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:35.896854 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:35.904895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:35.905353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:35.906130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:35.906563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:36.119230 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:36.133719 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:36.135742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:36.137182 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f1bc081a-71a9-4785-aa0d-bd659585710f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1751 microversion: 2.20 time: 0.250303 Jul 24 01:03:36.138195 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 195/380] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:35 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1751 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:36.444883 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:36.445893 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:36.452597 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:36.453251 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:36.454694 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:36.455309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:36.604471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:36.613313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e859ec31-dd62-4c9f-a249-210d02a0cf87 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.173159 Jul 24 01:03:36.614255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 186/381] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:36 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:37.155930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:37.156632 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:37.163457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:37.163990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:37.164997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:37.165633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:37.320728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:37.330029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:37.331576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:37.332977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fe5bf77c-c599-4eeb-a5bf-2ff5b8ad72a2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.180996 Jul 24 01:03:37.333700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 196/382] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:37 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:37.630394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:37.631629 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:37.634258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:37.642131 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:37.642582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:37.643391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:37.643832 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:37.797096 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:37.807700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5829c72f-dc82-4f5f-afce-ad852eaa5fd9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.181204 Jul 24 01:03:37.817845 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 187/383] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:37 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:38.351561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:38.352867 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:38.359425 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:38.360155 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:38.361293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:38.361953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:38.512293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:38.522139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:38.523838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:38.525025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a41190a5-6592-48b4-bbb7-9bc80753bf68 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.177370 Jul 24 01:03:38.525895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 197/384] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:38 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:38.829705 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:38.831142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:38.837997 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:38.838481 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:38.839290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:38.839894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:38.981612 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:38.990305 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-34fe4386-f728-4140-9517-1d99e23e0889 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.168886 Jul 24 01:03:38.991051 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 188/385] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:38 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:39.542975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:39.544308 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:39.561878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:39.562651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:39.566036 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:39.585514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:39.783278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:39.792798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:39.794294 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:39.795741 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-14b1c09a-ead7-4a1c-9bf7-3377a8096d6a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.256726 Jul 24 01:03:39.796597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 198/386] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:39 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:40.008028 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:40.008669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:40.015658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:40.016326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:40.017981 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:40.018420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:40.174764 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:40.188523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2d6a5116-8fa3-4fbf-a796-df595886e08d tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1368 microversion: 2.1 time: 0.184978 Jul 24 01:03:40.189589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 189/387] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:40 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1368 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:40.831528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:40.833235 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:40.841730 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:40.842188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:40.843016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:40.843643 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:40.993004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:41.003255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:41.005457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:41.006684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-42bfa055-86a6-4ad8-8835-8ecc2f436fb1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.193901 Jul 24 01:03:41.018316 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 199/388] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:40 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:41.206461 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:41.207106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:41.213525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:41.214046 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:41.214903 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:41.215521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:41.367619 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:41.391127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2491a73f-0aee-4d3f-adb8-94cd8aa14075 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1559 microversion: 2.1 time: 0.189355 Jul 24 01:03:41.393715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 190/389] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:41 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1559 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:41.411371 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:41.411963 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:41.424974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:41.425839 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:41.426950 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:41.427566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:41.524014 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-188d27c3-05ca-4f2a-9bea-935aedf51d18 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.117816 Jul 24 01:03:41.526807 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 200/390] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:03:41 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/os-interface => generated 285 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:41.528025 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-adf4e00a-203b-42ed-9396-48a68a15aae1 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Action: 'create', calling method: .version_select at 0x7f5e31268160>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1269427714", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO5P9t6RJ3xgW5WBwUSYTghhlzT3WZX3ypEGnPnnUjY0rBYBESH/crosnXv2EuEcoqUUFfNfKCY8zNCZfs9dx7UkZJy7EyEQXKVNOG60iTulb58AcrU63m3e/T7GRDQ72w=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:41.543116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-adf4e00a-203b-42ed-9396-48a68a15aae1 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Getting quotas for project ee9d5b852be444f48a33b2f4be5c34be. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:03:41.547536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-adf4e00a-203b-42ed-9396-48a68a15aae1 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Getting quotas for user ddda2bac031d4635b27421d22bb40fff and project ee9d5b852be444f48a33b2f4be5c34be. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:03:41.571813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-adf4e00a-203b-42ed-9396-48a68a15aae1 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.047528 Jul 24 01:03:41.572854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 191/391] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:03:41 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:03:41.636803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1348311314", "imageRef": "0fc72a18-06de-4927-a4d0-612f16b1db25", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1269427714", "networks": [{"uuid": "a03d2c45-4db4-45be-ae0a-38988e70c5b5"}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:41.689138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] validate_networks() for [('a03d2c45-4db4-45be-ae0a-38988e70c5b5', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:03:42.029088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:42.030147 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:42.039805 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:42.040836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:42.041820 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:42.043644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:42.433891 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:42.442602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:42.444423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:42.445605 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-bf90cc73-3768-4a78-afdf-a09260b7d047 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.423967 Jul 24 01:03:42.446347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 192/392] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:42 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:03:42.510599 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:03:42.511118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:03:42.511519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:03:42.511921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:03:42.512356 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:03:42.523046 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:42.523491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:42.524283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:42.524714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:42.542821 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Getting quotas for project ee9d5b852be444f48a33b2f4be5c34be. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:03:42.547860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Getting quotas for user ddda2bac031d4635b27421d22bb40fff and project ee9d5b852be444f48a33b2f4be5c34be. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:03:42.559070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:03:42.561326 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='0fc72a18-06de-4927-a4d0-612f16b1db25',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:03:42.612350 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-43bb06e6-3dec-427b-bd3c-724b7ff87e46 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.979605 Jul 24 01:03:42.612996 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 201/393] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:03:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 981 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 24 01:03:42.623850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-75cb4503-27fa-44e9-a05e-74fd35b6f692 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:42.624466 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-75cb4503-27fa-44e9-a05e-74fd35b6f692 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:42.977899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-75cb4503-27fa-44e9-a05e-74fd35b6f692 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:42.986061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-75cb4503-27fa-44e9-a05e-74fd35b6f692 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1259 microversion: 2.1 time: 0.366645 Jul 24 01:03:42.989213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 193/394] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:42 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1259 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:43.320193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-d0206153-ac95-4264-aafc-35cde3837ee2 req-36f0e4f3-ced0-4a8a-9df3-1c96d08625d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fd889446-7a8d-4e52-91a5-22b75a923a21", "tag": "0a201010-559e-40e9-8371-e03c2a2e2d4c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:43.332361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-d0206153-ac95-4264-aafc-35cde3837ee2 req-36f0e4f3-ced0-4a8a-9df3-1c96d08625d7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:43.340670 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-d0206153-ac95-4264-aafc-35cde3837ee2 req-36f0e4f3-ced0-4a8a-9df3-1c96d08625d7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:43.364293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-d0206153-ac95-4264-aafc-35cde3837ee2 req-36f0e4f3-ced0-4a8a-9df3-1c96d08625d7 service nova] Creating event network-changed:0a201010-559e-40e9-8371-e03c2a2e2d4c for instance fd889446-7a8d-4e52-91a5-22b75a923a21 on hv-cinder-maste Jul 24 01:03:43.373045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-d0206153-ac95-4264-aafc-35cde3837ee2 req-36f0e4f3-ced0-4a8a-9df3-1c96d08625d7 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059078 Jul 24 01:03:43.373891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 202/395] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:43.464960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:43.467638 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:43.487314 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:43.488058 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:43.490338 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:43.491292 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:43.692434 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:43.701259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:43.702795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:43.704218 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-74b136fb-caf8-4201-9f7f-8c42d0eb8f1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.243585 Jul 24 01:03:43.707972 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 194/396] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:43 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:43.814119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:43.814644 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:43.821285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:43.821740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:43.822514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:43.822998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:43.913733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lazy-loading 'services' on Instance uuid c28bad0d-af40-4706-8310-b7212eccb24e {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:03:43.996546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4fead6a3-f30d-4082-ba4d-11a969ac8319 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e/os-volume_attachments/c7517365-72cd-4b65-9480-378e0130c123" status: 202 len: 0 microversion: 2.1 time: 0.186874 Jul 24 01:03:43.997294 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 203/397] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:03:43 2022] DELETE /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e/os-volume_attachments/c7517365-72cd-4b65-9480-378e0130c123 => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:03:44.006029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:44.006835 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:44.013571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:44.014145 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:44.015153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:44.015680 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:44.122826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:44.133963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d147a2ee-a0fe-4022-bda9-3c6d9ed0e006 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.131977 Jul 24 01:03:44.135773 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 195/398] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:44 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:44.725290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:44.725945 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:44.732295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:44.732743 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:44.733543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:44.734090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:44.875377 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:44.884746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:44.886286 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:44.887592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-059cae1e-5c5b-4933-8076-bc59ab095ce8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.167922 Jul 24 01:03:44.891120 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 204/399] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:44 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:45.151800 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:45.152440 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:45.159229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:45.159681 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:45.160443 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:45.160878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:45.227115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:45.229323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=100 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:45.235840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a4841525-32e0-4d79-b08b-12eca79892ab tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.087858 Jul 24 01:03:45.236498 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 196/400] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:45 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:45.905492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:45.906727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:45.913086 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:45.913536 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:45.914307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:45.914746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:46.062395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:46.072331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:46.074106 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:46.075496 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f15cbd4f-6686-4881-aed2-7cf3996df862 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.173875 Jul 24 01:03:46.076263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 205/401] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:45 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:46.255439 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:46.256858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:46.264877 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:46.265320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:46.266118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:46.266720 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:46.329267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:46.337128 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2a45e40b-289f-400d-b1fe-0bfeb7eef220 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.087399 Jul 24 01:03:46.337712 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 197/402] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:46 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:47.094340 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:47.097431 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:47.104107 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:47.104716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:47.105400 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:47.106000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:47.276157 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:47.285565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:47.288413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:47.289781 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5485db75-2758-4924-a327-8a1a8a33d9d5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.199788 Jul 24 01:03:47.290908 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 206/403] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:47 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:47.354696 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:47.355385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:47.361743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:47.362191 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:47.363007 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:47.363450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:47.429160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:47.437735 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a34af131-b5e9-4a7d-98a7-e796a851e499 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.086936 Jul 24 01:03:47.439197 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 198/404] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:47 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:48.308920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:48.310563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:48.316966 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:48.317849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:48.319309 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:48.319967 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:48.455944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:48.456623 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:48.468444 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:48.468897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:48.470488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:48.470960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:48.477795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:48.487546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:48.489393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:48.490887 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-efca1a70-d3bc-437c-be5f-dd50ced58ec6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.186188 Jul 24 01:03:48.492307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 207/405] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:48 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:48.541506 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:48.549401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a72d9a59-72e4-440e-b80b-d8a0a0226296 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.098148 Jul 24 01:03:48.550003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 199/406] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:48 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:49.509478 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:49.510221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:49.516557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:49.517015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:49.517927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:49.518369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:49.521354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:49.566701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:49.567400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:49.574256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:49.574708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:49.575567 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:49.576005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:49.663979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:49.671633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-df2d699d-1bc8-4a13-b7d6-8b96b860f13b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.108885 Jul 24 01:03:49.672207 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 200/407] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:49 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:49.692865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:49.701446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:49.702962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:49.704176 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5fba3eb9-984b-40fb-9da2-881074c49e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.198622 Jul 24 01:03:49.704732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 208/408] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:49 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:50.689225 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:50.689814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:50.696413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:50.696863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:50.697617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:50.698046 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:50.724473 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:50.728830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:50.735794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:50.736718 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:50.737738 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:50.738398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:50.783076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:50.790829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-870f914e-20c3-451b-8bb3-5d28e005ede3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.105687 Jul 24 01:03:50.791397 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 201/409] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:50 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:50.881847 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:50.892584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:50.894412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:50.895930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4fac28a8-604f-40d1-936d-6f8bb5fe236b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.176409 Jul 24 01:03:50.896835 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 209/410] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:50 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:51.807589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:51.808198 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:51.815346 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:51.815795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:51.816560 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:51.816997 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:51.877104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:51.885765 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e772aa98-77cb-43d5-9889-039664efc45f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1263 microversion: 2.1 time: 0.082075 Jul 24 01:03:51.886347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 202/411] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:51 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1263 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:51.913414 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:51.913999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:51.919985 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:51.920438 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:51.921205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:51.921647 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:52.053885 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:52.062807 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:52.064899 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:52.066090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-94615a39-26c7-4562-964b-7abdd07b63b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.156558 Jul 24 01:03:52.066652 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 210/412] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:51 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:52.903552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:52.904166 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:52.911400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:52.911846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:52.912614 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:52.913052 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:52.973766 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:52.981184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ca7de490-af3e-4bac-8853-fbd1e02ea3d4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1255 microversion: 2.1 time: 0.081545 Jul 24 01:03:52.981741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 203/413] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:52 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1255 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:53.085159 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:53.085771 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:53.087745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:53.088398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:53.094984 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:53.095532 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:53.096308 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:53.096749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:53.226271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:53.234629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:53.236673 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:53.237986 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-96a16534-0aeb-4fe1-92d2-913040502de3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.156696 Jul 24 01:03:53.238736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 211/414] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:53 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:53.998069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:53.999366 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:54.001462 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:54.007767 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.008507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.009460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.009982 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.082698 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:54.090523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fe31c909-2b7a-4bdc-9100-77a6d7b3e1a8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1311 microversion: 2.1 time: 0.096351 Jul 24 01:03:54.096713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 204/415] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:53 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1311 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:54.256865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:54.257479 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:54.263692 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.264139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.264909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.265344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.403870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:54.414428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:54.416092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:54.417493 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-552a0e2d-cf7c-40da-8734-f11c0f0f5dfd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.164513 Jul 24 01:03:54.425189 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 212/416] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:54 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:54.693442 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:54.694208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:54.701229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.701685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.702465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.702932 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.736507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:03:54.818645 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-327282d3-21cf-4d81-acae-7314a934616b tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 204 len: 0 microversion: 2.1 time: 0.128765 Jul 24 01:03:54.819528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 205/417] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:03:54 2022] DELETE /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:03:54.828502 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:54.829311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:54.838651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.839302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:54.840096 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:54.840783 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:55.038029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:55.045561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b3c173f2-a23b-4a15-9b04-107ef61439b4 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1512 microversion: 2.1 time: 0.221775 Jul 24 01:03:55.046162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 213/418] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:54 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1512 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:55.108811 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:55.109507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:55.116101 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:55.116552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:55.117308 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:55.117768 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:55.179318 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:55.187018 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-303d212e-88f6-4611-bf3f-9c7d2a542fad tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1329 microversion: 2.1 time: 0.082341 Jul 24 01:03:55.187597 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 206/419] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:55 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1329 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:55.436002 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:55.436641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:55.443067 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:55.443520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:55.444547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:55.444982 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:55.574137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:55.582821 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:55.584336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:55.585495 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6b907525-0daa-4ee6-b072-a0af40ef1023 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.153437 Jul 24 01:03:55.586058 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 214/420] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:55 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:56.062523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:56.063269 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:56.069713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:56.070154 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:56.070945 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:56.071376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:56.197254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:56.204920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:56.205533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:56.207041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a2d83f2d-9e1e-4ca3-baad-8fe60a70e251 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1512 microversion: 2.1 time: 0.148335 Jul 24 01:03:56.207613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 207/421] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:56 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1512 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:56.211712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:56.212167 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:56.212934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:56.213373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:56.273363 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:56.280713 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-84df924f-36b7-468f-bd5e-7026a3535e7f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1329 microversion: 2.1 time: 0.080292 Jul 24 01:03:56.281276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 215/422] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:56 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1329 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:56.604268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:56.604898 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:56.612053 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:56.612538 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:56.613642 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:56.614235 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:56.788370 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:56.800980 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:56.802821 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:56.803709 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9d39f638-3abe-4319-b6fd-d1af78e48a4c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.203347 Jul 24 01:03:56.804623 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 208/423] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:56 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:57.224923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:57.225541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:57.232212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:57.232668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:57.233450 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:57.233889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:57.300697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:57.301706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:57.332068 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:57.332531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:57.341095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:57.342385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:57.933370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:57.936376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:57.946902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-131668ac-da49-453d-8681-b5a0e57e1cd3 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1512 microversion: 2.1 time: 0.725717 Jul 24 01:03:57.948217 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 216/424] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:57 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1512 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:57.957181 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:57.957370 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e5198ead-025e-4fc3-b919-2b048204fd6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.663470 Jul 24 01:03:57.966789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 209/425] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:57 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:03:57.966789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-c6017fa8-0eb3-422c-bd81-29085e33c26a req-9ebc9672-5e57-4302-8fb6-22b5c8dcce04 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c28bad0d-af40-4706-8310-b7212eccb24e", "name": "network-vif-deleted", "tag": "260c7eb5-b1ed-40d9-b311-c9db67b1cfaa"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:57.967618 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:57.977650 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-c6017fa8-0eb3-422c-bd81-29085e33c26a req-9ebc9672-5e57-4302-8fb6-22b5c8dcce04 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:57.980875 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-c6017fa8-0eb3-422c-bd81-29085e33c26a req-9ebc9672-5e57-4302-8fb6-22b5c8dcce04 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:57.984369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:57.985762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:57.999890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:58.000768 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:58.008936 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-c6017fa8-0eb3-422c-bd81-29085e33c26a req-9ebc9672-5e57-4302-8fb6-22b5c8dcce04 service nova] Creating event network-vif-deleted:260c7eb5-b1ed-40d9-b311-c9db67b1cfaa for instance c28bad0d-af40-4706-8310-b7212eccb24e on hv-cinder-maste Jul 24 01:03:58.013842 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-c6017fa8-0eb3-422c-bd81-29085e33c26a req-9ebc9672-5e57-4302-8fb6-22b5c8dcce04 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053957 Jul 24 01:03:58.014508 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 210/426] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:58.236173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:58.245420 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:58.251822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:58.261552 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b50b53b4-4b73-468c-9180-2030c5d22422 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.312140 Jul 24 01:03:58.262244 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 217/427] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:57 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:03:58.511819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-3cbeb669-08e7-41bc-8422-46811c3d3958 req-8e55f81e-9f50-4af2-b199-5ac04878bba7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:58.535171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3cbeb669-08e7-41bc-8422-46811c3d3958 req-8e55f81e-9f50-4af2-b199-5ac04878bba7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:58.535748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3cbeb669-08e7-41bc-8422-46811c3d3958 req-8e55f81e-9f50-4af2-b199-5ac04878bba7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:58.547634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-3cbeb669-08e7-41bc-8422-46811c3d3958 req-8e55f81e-9f50-4af2-b199-5ac04878bba7 service nova] Creating event network-changed:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:03:58.552588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-3cbeb669-08e7-41bc-8422-46811c3d3958 req-8e55f81e-9f50-4af2-b199-5ac04878bba7 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045562 Jul 24 01:03:58.553493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 211/428] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:03:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:03:58.965036 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:58.965804 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:58.973139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:58.973795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:58.974801 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:58.975342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:58.976196 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:58.977084 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:58.983927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:58.984658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:58.985930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:58.986543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:59.051260 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:59.059922 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e4c439d4-3d8e-47e2-bf93-8cb35cdf81ba tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.098657 Jul 24 01:03:59.061234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 218/429] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:58 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:59.068373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d1a3d8e7-19a4-40bf-a311-17e785b4bf50"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:03:59.069795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:59.076191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:59.076647 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:59.077425 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:59.077890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:59.140686 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:59.149794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a3cebe9e-be0f-4486-8081-236a886d4c9d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.179577 Jul 24 01:03:59.151128 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 212/430] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:03:58 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:03:59.280396 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:03:59.281026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:03:59.288487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:59.289041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:59.290159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:03:59.290609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:03:59.293718 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:03:59.450711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:59.460878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:59.462394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:03:59.463777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-db49ab59-4327-44c7-9b85-5c6aad35da5b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.187425 Jul 24 01:03:59.464515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 213/431] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:03:59 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:00.078575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:00.079354 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:00.085664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:00.086120 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:00.086909 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:00.087345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:00.148598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:00.156032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-66b0d193-7d0f-45c0-957a-0a8cce87e341 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.081532 Jul 24 01:04:00.156636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 214/432] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:00 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:00.166610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:00.167243 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:00.174038 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:00.174488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:00.175297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:00.175734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:00.322624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:00.331759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6c32f61a-d4d5-4e56-9c85-cf76edf5d2ef tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.168710 Jul 24 01:04:00.332595 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 215/433] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:00 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:00.485930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:00.486546 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:00.493790 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:00.494734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:00.495781 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:00.496476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:00.639437 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:00.651707 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:00.653459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:00.655500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0e837ef7-25f9-4892-9a55-c54ae9445d71 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.173756 Jul 24 01:04:00.657447 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 216/434] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:00 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:01.173719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:01.174447 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:01.181363 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:01.181904 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:01.182683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:01.183143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:01.250796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:01.259032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-68942ac0-214d-4751-bee3-2a1b4d5fe3dd tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.089798 Jul 24 01:04:01.259610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 217/435] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:01 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:01.349829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:01.350433 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:01.356980 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:01.357438 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:01.358217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:01.358657 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:01.493277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:01.500941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8bf6754a-0cd2-44e9-9cdb-5f3f37fb629b tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.155062 Jul 24 01:04:01.501509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 218/436] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:01 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:01.676130 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:01.676658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:01.682848 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:01.683305 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:01.684074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:01.684512 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:01.829092 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:01.838171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:01.839728 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:01.840902 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5904f1fa-a161-4555-9c59-b4e5e948b8a9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.168854 Jul 24 01:04:01.841663 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 219/437] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:01 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:02.277378 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:02.278107 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:02.285576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:02.286026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:02.286901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:02.287440 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:02.360122 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:02.368395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3c94a45f-08eb-4a74-b55b-6fe2c6834501 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.094839 Jul 24 01:04:02.368983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 220/438] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:02 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:02.518699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:02.519481 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:02.525675 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:02.526117 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:02.526917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:02.527349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:02.658731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:02.671946 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3a412211-d9fe-4f67-a0f3-8b124c7b6346 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.157589 Jul 24 01:04:02.672932 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 221/439] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:02 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:02.860737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:02.861300 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:02.867407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:02.867852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:02.868611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:02.869045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:03.021830 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:03.031258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:03.032884 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:03.034570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a343481b-2c96-425c-a6a7-18264931da3a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.178244 Jul 24 01:04:03.035350 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 222/440] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:02 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:03.385950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:03.386638 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:03.393340 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:03.393797 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:03.394592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:03.395060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:03.476675 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:03.485324 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6f8a99f5-ba25-41a4-98fd-9cd003244f9f tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.102774 Jul 24 01:04:03.494735 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 223/441] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:03 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:03.689030 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:03.689555 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:03.696042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:03.696526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:03.697293 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:03.697740 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:03.839482 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:03.847977 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-44e45490-9879-410b-8016-7479390932a3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.162923 Jul 24 01:04:03.848544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 224/442] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:03 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:04.053504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:04.054028 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:04.060528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:04.060991 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:04.061807 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:04.062251 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:04.198290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:04.208757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:04.211945 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:04.212117 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e96661dd-a3cf-4e14-96b0-c027deeb8f63 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.162530 Jul 24 01:04:04.212761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 225/443] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:04 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:04.502309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:04.503050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:04.510006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:04.510558 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:04.511446 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:04.511952 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:04.578492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:04.586173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fc4ddc24-5643-45d7-a6a6-5e9519c8037e tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.088408 Jul 24 01:04:04.586745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 226/444] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:04 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:04.652005 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:04:04.652627 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:04:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments (ip 10.222.0.18) !!! Jul 24 01:04:04.865575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:04.866179 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:04.872436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:04.872885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:04.873654 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:04.874081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:05.016943 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:05.024378 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9fdac063-e61d-49fe-99c9-6002bc896155 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.162901 Jul 24 01:04:05.025680 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 227/445] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:04 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:05.231609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:05.232246 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:05.234111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:05.239476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:05.239921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:05.240768 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:05.241209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:05.389995 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:05.399367 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:05.400855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:05.402001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f9d66905-5538-4d23-80b6-d4afe79cb9a4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.174411 Jul 24 01:04:05.402560 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 228/446] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:05 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:05.605133 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:05.606286 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:05.613404 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:05.613786 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:05.614577 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:05.615016 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:05.681685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:05.683959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=100 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:05.694443 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d478f389-d186-46d6-8693-2ffa0ef185a6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.094387 Jul 24 01:04:05.697399 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 229/447] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:05 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:06.043183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:06.044295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:06.051079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.051537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.052328 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.052767 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.186473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:06.194896 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d000fa3b-4bab-44eb-9126-9b7d0766010e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.155703 Jul 24 01:04:06.195464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 230/448] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:06 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:06.420990 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:06.421591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:06.428284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.428730 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.429483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.429915 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.555990 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:06.564308 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:06.565809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:06.567004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5a8a131f-a8b2-4437-96bb-7031ab523cc6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.149916 Jul 24 01:04:06.567558 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 231/449] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:06 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:06.572315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:06.572801 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:06.578542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.579019 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.579778 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.580208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.702201 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lazy-loading 'services' on Instance uuid 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:04:06.814290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5d91d08e-7ba3-4d13-9110-496037ddb305 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-volume_attachments/3abfea65-7ffe-41de-a680-bd5a8ed5ccdc" status: 202 len: 0 microversion: 2.1 time: 0.245543 Jul 24 01:04:06.814913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 232/450] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:04:06 2022] DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-volume_attachments/3abfea65-7ffe-41de-a680-bd5a8ed5ccdc => generated 0 bytes in 247 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:04:06.820188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:06.820789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:06.826968 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.827419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.828350 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:06.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:06.898224 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:06.905899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a27ee120-dd04-4b6d-a360-cba9eda76e7d tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1470 microversion: 2.1 time: 0.089888 Jul 24 01:04:06.906703 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 233/451] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:06 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1470 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:07.213051 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:07.213659 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:07.220549 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.220992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.221743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.222174 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.373387 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:07.382713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-28cf857b-1516-4e87-a93e-1a82066e8308 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.173414 Jul 24 01:04:07.383775 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 234/452] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:07 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:07.586011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:07.586539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:07.592803 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.593253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.594602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.595071 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.733947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:07.743196 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:07.744695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:07.745840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-19764d27-fc72-494f-948c-f1ae981b4edb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.163936 Jul 24 01:04:07.746408 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 235/453] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:07 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:07.924777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:07.925788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:07.935006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.936010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.938009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.938465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.977287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:07.977837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:07.978347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lazy-loading 'fault' on Instance uuid c28bad0d-af40-4706-8310-b7212eccb24e {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:04:08.023972 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1ab7da65-2316-4475-bb83-0f2245919dae tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 8.961482 Jul 24 01:04:08.026977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 219/454] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:03:59 2022] POST /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments => generated 194 bytes in 8965 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:08.058438 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e4b9bb864f24c93b2c9547432b8fd96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4dd26bb2797345d6b531f5ba5db5cbc8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:08.067445 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-29cc9c30-3db4-4b10-8512-39f8b6f1cda6 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 200 len: 1309 microversion: 2.1 time: 0.148011 Jul 24 01:04:08.077371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 236/455] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:07 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 1309 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:08.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:08.405476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:08.408294 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:08.414160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:08.414620 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:08.415436 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:08.415879 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:08.683539 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:08.691908 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b3ae15ea-087a-46e3-a278-2028b1693e3e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.291985 Jul 24 01:04:08.692484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 220/456] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:08 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:08.765225 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:08.766447 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:08.773070 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:08.773520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:08.774400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:08.774864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:08.982960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:08.992201 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:08.993786 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:08.995186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-04ccaa10-d8de-4c65-9553-1d756c71bfc9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.234229 Jul 24 01:04:09.007066 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 237/457] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:08 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:09.085165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:09.085766 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] [instance: c28bad0d-af40-4706-8310-b7212eccb24e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:09.091921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:09.092373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:09.093134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:09.093629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:09.107451 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] HTTP exception thrown: Instance c28bad0d-af40-4706-8310-b7212eccb24e could not be found. Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Request method failure captured: Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e HTTP/1.1 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnjXx2dOBLlVMg0fmFkzZItJuSIa81Gw3chUF7SqSm6RRmy6iQSnm3Ld0Id92t0S_UNzg_lFrC2CEk77o7zqqCltZy9KUcVcwZh10Fms4RgrniL5S8EPtPeFtDlO7tsT3QTj_DMOEdZ3MNeswWkakQBNUrVUjeNGHQUrvfXkeOLFV0 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 4dd26bb2797345d6b531f5ba5db5cbc8 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesSnapshotTestJSON-425251886 Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:04:09.109090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/4dd26bb2797345d6b531f5ba5db5cbc8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/4dd26bb2797345d6b531f5ba5db5cbc8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/4dd26bb2797345d6b531f5ba5db5cbc8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_4dd26bb2797345d6b531f5ba5db5cbc8", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesSnapshotTestJSON-425251886 Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 4dd26bb2797345d6b531f5ba5db5cbc8 Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-425251886 Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesSnapshotTestJSON-425251886-project Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 0e4b9bb864f24c93b2c9547432b8fd96 Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesSnapshotTestJSON-425251886-project Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance c28bad0d-af40-4706-8310-b7212eccb24e could not be found. Jul 24 01:04:09.111520 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'c28bad0d-af40-4706-8310-b7212eccb24e'} Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c28bad0d-af40-4706-8310-b7212eccb24e could not be found. Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:04:09.115844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c28bad0d-af40-4706-8310-b7212eccb24e could not be found. Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c28bad0d-af40-4706-8310-b7212eccb24e could not be found. Jul 24 01:04:09.117063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:09.118137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] Returning 404 to user: Instance c28bad0d-af40-4706-8310-b7212eccb24e could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:04:09.118137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-70aa2aef-4605-48d2-a57f-c8410bb79923 tempest-VolumesSnapshotTestJSON-425251886 tempest-VolumesSnapshotTestJSON-425251886-project] 10.222.0.18 "GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e" status: 404 len: 111 microversion: 2.1 time: 0.030223 Jul 24 01:04:09.118137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 221/458] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:09 2022] GET /compute/v2.1/servers/c28bad0d-af40-4706-8310-b7212eccb24e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:04:09.708592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:09.709233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:09.715930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:09.716377 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:09.717138 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:09.717574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:09.849723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:09.857460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2c4f57c0-e433-407d-8309-1b95a1b6e632 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.152759 Jul 24 01:04:09.858049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 238/459] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:09 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:10.014788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:10.015916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:10.022744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.023251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:10.024081 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.024518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:10.091962 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-84a1f2ef-5f6a-4596-95c2-df4ef887142a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:10.099143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-84a1f2ef-5f6a-4596-95c2-df4ef887142a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.099966 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-84a1f2ef-5f6a-4596-95c2-df4ef887142a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:10.170320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-84a1f2ef-5f6a-4596-95c2-df4ef887142a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Going to try to stop instance {{(pid=117956) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 24 01:04:10.179875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:10.190805 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:10.193219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:10.194678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9fc6734b-9994-4358-b3fd-a885ff66646b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.183923 Jul 24 01:04:10.204298 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 222/460] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:10 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:10.222489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-84a1f2ef-5f6a-4596-95c2-df4ef887142a tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action" status: 202 len: 0 microversion: 2.1 time: 0.134853 Jul 24 01:04:10.223132 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 239/461] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:04:10 2022] POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action => generated 0 bytes in 137 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:04:10.231289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:10.231907 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:10.238850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.239307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:10.240084 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.240521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:10.373932 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:10.381559 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2027d8d0-41e7-442a-942c-6c403d5e28f3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.154093 Jul 24 01:04:10.382128 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 223/462] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:10 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:10.876374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:10.876982 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:10.883173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.883626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:10.884407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:10.884837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:11.016083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:11.023514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-020871b7-a397-4549-880e-9629fbd6e4d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.152529 Jul 24 01:04:11.024079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 240/463] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:10 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:11.214253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:11.214879 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:11.221680 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:11.222127 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:11.222908 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:11.223340 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:11.225942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:11.355462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:11.364328 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:11.365830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:11.367020 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bff5352c-325d-4c22-a396-60d148818916 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.156578 Jul 24 01:04:11.367585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 224/464] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:11 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:11.398565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:11.399204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:11.405298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:11.405748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:11.406673 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:11.407137 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:11.534210 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:11.541690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7a5ad3a9-880a-4031-9e29-f4c2dc4ef152 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.146952 Jul 24 01:04:11.542248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 241/465] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:11 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:12.039818 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:12.040440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:12.047078 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:12.047695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:12.048456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:12.048888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:12.175214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:12.182599 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bd61bec8-07e0-4fa4-993a-cd351e48976e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.146651 Jul 24 01:04:12.183195 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 225/466] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:12 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:12.386602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:12.387227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:12.394397 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:12.394867 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:12.395635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:12.396074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:12.526173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:12.534646 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:12.536161 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:12.537299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-61c4f88f-eefb-454b-bbe6-03d1899f189b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.154592 Jul 24 01:04:12.538091 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 242/467] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:12 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:12.559348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:12.559888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:12.567299 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:12.567743 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:12.568498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:12.569203 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:12.697008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:12.704432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-14ee67e9-0ba4-4f96-af65-7b53d1477a44 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.148979 Jul 24 01:04:12.704986 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 226/468] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:12 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:13.199483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:13.200095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:13.206265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.206715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.207575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.208013 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.338376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:13.346299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7f2310c8-2922-4694-a65e-f57687cf5046 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.150649 Jul 24 01:04:13.346890 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 243/469] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:13 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:13.555916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:13.556526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:13.563486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.563998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.564700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.565136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.701851 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:13.711683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:13.713962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:13.715390 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-90efd823-da68-486e-bc2e-48245bf830c6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.163407 Jul 24 01:04:13.716019 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 227/470] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:13 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:13.735230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:13.735684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-8b628150-d78c-4dc3-9e0f-0014f441ddff req-074a5a3f-1eb3-4b9a-8e76-bb369b46c5b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "name": "network-vif-plugged", "status": "completed", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:13.736065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:13.743974 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.744439 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.745208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.745482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-8b628150-d78c-4dc3-9e0f-0014f441ddff req-074a5a3f-1eb3-4b9a-8e76-bb369b46c5b6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:13.745846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.745981 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-8b628150-d78c-4dc3-9e0f-0014f441ddff req-074a5a3f-1eb3-4b9a-8e76-bb369b46c5b6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:13.758190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-8b628150-d78c-4dc3-9e0f-0014f441ddff req-074a5a3f-1eb3-4b9a-8e76-bb369b46c5b6 service nova] Creating event network-vif-plugged:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:04:13.763185 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-8b628150-d78c-4dc3-9e0f-0014f441ddff req-074a5a3f-1eb3-4b9a-8e76-bb369b46c5b6 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033574 Jul 24 01:04:13.763387 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:13.763807 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 228/471] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:13.882691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:13.890098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8a751279-af27-4c8d-a511-dc3b47e80dd9 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.160706 Jul 24 01:04:13.891556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 244/472] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:13 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:14.364551 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:14.365260 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:14.371460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:14.371915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:14.372686 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:14.373122 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:14.530299 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:14.538302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-77049c86-058b-46b3-b329-aa78c1a7fe96 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.178012 Jul 24 01:04:14.538885 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 229/473] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:14 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:14.736521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:14.737147 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:14.743745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:14.744192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:14.744950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:14.745380 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:14.879316 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:14.887886 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:14.889374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:14.891188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-46202c01-87d8-4a70-af76-97c24427894e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.158529 Jul 24 01:04:14.891769 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 245/474] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:14 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:14.907968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:14.908617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:14.915386 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:14.915914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:14.916681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:14.917114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:15.065460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:15.073243 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e08f72a9-9dd9-4266-a3c1-470bb1f21be4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.169430 Jul 24 01:04:15.074035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 230/475] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:14 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:15.554916 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:15.555517 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:15.561507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:15.562471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:15.563541 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:15.563977 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:15.698733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:15.706374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fb6ee93f-c2bd-4f33-ab6a-a7362f4ce532 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.155212 Jul 24 01:04:15.707050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 246/476] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:15 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:15.909611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:15.910219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:15.916744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:15.917199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:15.917957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:15.918393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:16.048021 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:16.056970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:16.058464 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:16.059653 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9d60d4d4-92b7-49a4-ad70-2ccd604cb855 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.154102 Jul 24 01:04:16.060210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 231/477] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:15 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:16.091808 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:16.092518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:16.099478 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:16.100014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:16.100917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:16.101431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:16.243965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:16.252302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a1d14e17-fd26-4f52-92d9-633ac5e876a3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.164275 Jul 24 01:04:16.252880 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 247/478] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:16 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:16.724435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:16.725134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:16.731283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:16.731727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:16.732483 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:16.732912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:16.872160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:16.879705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-19c58dce-06f3-4391-8887-36dfedd16d54 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.159053 Jul 24 01:04:16.880342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 232/479] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:16 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:17.078674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:17.079590 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:17.086645 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.087116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.087892 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.088327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.232287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:17.242359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:17.244195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:17.245762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6e138d9a-2f6d-410d-a5d0-a6e4a460751f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1807 microversion: 2.20 time: 0.171437 Jul 24 01:04:17.246507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 248/480] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:17 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1807 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:04:17.268770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:17.269514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:17.276934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.277393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.278488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.279031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.439481 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:17.448856 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-610c62ec-7793-45af-9698-4455ac1da792 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.184009 Jul 24 01:04:17.449431 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 233/481] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:17 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:17.676416 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:17.677462 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:17.683592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.684039 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.684795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.685224 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.708135 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lazy-loading 'flavor' on Instance uuid c6a71fa6-32c2-446f-b9e5-80d4e9ea800e {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:04:17.775118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-a3e175f1-b142-42b7-89a9-28c391ae1413 req-04098912-1af2-4bce-8262-074d58b98fb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "name": "network-vif-plugged", "status": "completed", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:17.785586 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-a3e175f1-b142-42b7-89a9-28c391ae1413 req-04098912-1af2-4bce-8262-074d58b98fb8 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.786031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-a3e175f1-b142-42b7-89a9-28c391ae1413 req-04098912-1af2-4bce-8262-074d58b98fb8 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.809029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-a3e175f1-b142-42b7-89a9-28c391ae1413 req-04098912-1af2-4bce-8262-074d58b98fb8 service nova] Creating event network-vif-plugged:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:04:17.814100 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-a3e175f1-b142-42b7-89a9-28c391ae1413 req-04098912-1af2-4bce-8262-074d58b98fb8 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042558 Jul 24 01:04:17.815299 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 234/482] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:04:17.849862 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e7b5b579-5b47-4e53-b0d6-a4fca02aa0d6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/action" status: 200 len: 32 microversion: 2.1 time: 0.178849 Jul 24 01:04:17.850532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 249/483] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:04:17 2022] POST /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/action => generated 32 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:04:17.862677 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:17.863438 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:17.871351 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.871895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.872971 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.873514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.897626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:17.898483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:17.904970 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.905429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:17.906809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:17.907354 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:18.164255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:18.172487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-421939d8-c9b8-4904-90aa-1525f094b492 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.316246 Jul 24 01:04:18.173134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 235/484] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:17 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:18.187215 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:18.194865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ce45ce2f-f7e8-4f1f-87b1-c22953fd9116 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.301549 Jul 24 01:04:18.195429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 250/485] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:17 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:18.265865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:18.266464 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:18.272501 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:18.273046 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:18.273700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:18.274137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:18.467969 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:18.468684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:18.474702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:18.475190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:18.475959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:18.476392 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:18.480582 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:18.490221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:18.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:18.493154 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b5fe3784-6e2b-4dce-9324-fdd1ed52fd4d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1794 microversion: 2.20 time: 0.231721 Jul 24 01:04:18.493797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 236/486] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:04:18 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1794 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:04:18.603489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:18.610996 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9055ee57-766b-4d70-89ec-9701f47ead2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.148082 Jul 24 01:04:18.611570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 251/487] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:18 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:19.189667 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:19.190288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:19.196621 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:19.197077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:19.197846 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:19.198290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:19.211346 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:19.211959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:19.219493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:19.220005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:19.220785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:19.221221 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:19.396199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:19.402933 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:19.404253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-13054431-264a-4d6e-b860-0b90cc2313ff tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.218418 Jul 24 01:04:19.404990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 237/488] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:19 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:19.411293 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-eff2b3d9-3195-42ac-ac2e-1624d2b1362f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.203744 Jul 24 01:04:19.411863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 252/489] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:19 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:19.628172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:19.628697 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:19.634720 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:19.635199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:19.635955 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:19.636385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:19.762976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:19.771053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4d57020c-1139-45a8-8ddf-4c2719c6998b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.147000 Jul 24 01:04:19.771619 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 238/490] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:19 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:20.422810 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:20.423424 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:20.428080 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:20.428690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:20.430042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:20.430502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:20.431306 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:20.431745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:20.435143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:20.435592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:20.436351 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:20.437443 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:20.623544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:20.626404 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:20.632063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2efbb898-cc5b-46af-9191-015459074fa0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.213809 Jul 24 01:04:20.632724 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 253/491] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:20 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:20.633998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-dae3592e-38d0-4080-a47d-57a74475b754 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.209751 Jul 24 01:04:20.634704 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 239/492] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:20 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:20.787973 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:20.788662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:20.794673 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:20.795155 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:20.795914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:20.797044 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:20.925180 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:20.932600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a4302a81-9b54-4d6d-8ecf-a14a31ab879d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.148541 Jul 24 01:04:20.933159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 254/493] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:20 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:21.649455 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:21.649983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:21.650361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:21.650531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:21.656873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:21.657320 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:21.657658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:21.658078 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:21.658177 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:21.658510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:21.658884 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:21.659347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:21.850367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:21.854332 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:21.858348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0dcf13e5-9859-42ac-af9c-f07302736e06 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.212783 Jul 24 01:04:21.859657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 240/494] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:21 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:21.862287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6e9ff33c-1185-4918-9d82-80e45a19be4d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1359 microversion: 2.1 time: 0.216369 Jul 24 01:04:21.863053 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 255/495] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:21 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1359 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:21.949599 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:21.950204 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:21.956279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:21.956726 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:21.957482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:21.957914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:22.084722 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:22.092150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-979ffa47-d082-4ef2-ad8c-ae220f291783 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.146407 Jul 24 01:04:22.092723 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 241/496] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:21 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:22.878151 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:22.878754 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:22.879478 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:22.880090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:22.886570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:22.886798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:22.887080 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:22.887214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:22.887856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:22.887973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:22.888321 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:22.888442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:22.893061 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=100 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:23.091661 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:23.095563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:23.100571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-90b86031-1f45-44f6-a917-e8093c543e4d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.227222 Jul 24 01:04:23.102836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 256/497] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:22 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:23.108686 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-992741e6-9f97-41eb-a140-258a08c4d0b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1549 microversion: 2.1 time: 0.232177 Jul 24 01:04:23.109800 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:23.117695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 242/498] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:22 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1549 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:23.118862 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:23.131283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:23.132345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:23.138601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.139147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.139944 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.140449 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.152917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.153471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.154413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.154941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.345584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:23.354285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5d5cabb3-e556-4418-ae8c-229068bf01c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1549 microversion: 2.1 time: 0.227274 Jul 24 01:04:23.354883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 243/499] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:23 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1549 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:23.363006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:23.369797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:23.371507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4578d595-0902-4613-b73b-31fb2db01691 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.266143 Jul 24 01:04:23.372079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 257/500] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:23 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:23.373357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:23.374135 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Searching by: {'deleted': False, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be'} {{(pid=117955) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 24 01:04:23.380844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.381296 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.382089 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.382526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.473001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.multi_cell_list [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117955) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 24 01:04:23.568494 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-95f1e3ba-e692-414a-b6ee-c93de32abc80 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.202236 Jul 24 01:04:23.569254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 244/501] 10.222.0.18 () {58 vars in 1124 bytes} [Sun Jul 24 01:04:23 2022] GET /compute/v2.1/servers => generated 343 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:23.579950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:23.580556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:23.586785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.587250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.588011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:23.588454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:23.714996 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:23.722451 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-135e03e3-1383-47cb-ab0a-a484b7cf7103 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1549 microversion: 2.1 time: 0.146876 Jul 24 01:04:23.723050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 258/502] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:23 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1549 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:24.117431 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:24.118041 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:24.124642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:24.125709 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:24.126530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:24.126997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:24.389672 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:24.391537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:24.398869 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:24.399693 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:24.400990 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:24.401609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:24.406686 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:24.546988 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:24.555850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3af831ed-2a40-4ab8-adf8-d878cf96c1f9 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.442302 Jul 24 01:04:24.556423 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 245/503] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:24 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:24.591304 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:24.599444 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-47c3c003-5fb2-4faa-9f03-6e7ebeb9ac52 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.213627 Jul 24 01:04:24.600009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 259/504] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:24 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:25.225429 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ec0df126-d8c4-43b0-8666-778bb2276d7e"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:25.226516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:25.232894 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:25.233338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:25.234089 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:25.234517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:25.497743 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b2334b81-1265-475c-b340-7ef59db6e3bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.275988 Jul 24 01:04:25.498380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 246/505] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:04:25 2022] POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments => generated 194 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:25.572169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:25.572775 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:25.580623 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:25.581286 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:25.582227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:25.583127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:25.618330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:25.619061 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:25.626856 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:25.627317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:25.628097 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:25.628537 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:25.770920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:25.778819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ab176bea-9cb0-4b8a-afc2-bf0bd90dc2f2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.210708 Jul 24 01:04:25.779606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 260/506] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:25 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:25.799125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:25.807340 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f5a09175-3509-4c81-a1fe-d80037d22969 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.193633 Jul 24 01:04:25.807909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 247/507] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:25 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:26.798185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:26.798849 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:26.805210 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:26.805654 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:26.806407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:26.806862 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:26.823637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:26.824248 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:26.831679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:26.832049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:26.832935 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:26.833416 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:27.008436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:27.018439 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-08d0191d-7ab9-41f8-9e18-2d6ad8175ae6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.224295 Jul 24 01:04:27.029402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 261/508] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:26 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:27.030596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:27.038791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0ce53510-e7e7-4bab-b2b4-c16953b9b109 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1678 microversion: 2.1 time: 0.218871 Jul 24 01:04:27.039364 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 248/509] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:26 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1678 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:28.037616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:28.038150 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:28.045594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.046041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.046831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.047979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.055289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:28.055951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:28.063201 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.063757 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.064584 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.064978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.251319 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:28.253575 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:28.260141 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-299fe9b7-ccb5-4bc5-99d8-cf0297ff9b80 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.226386 Jul 24 01:04:28.260743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 262/510] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:28 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:28.261047 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-643f9301-c5e3-4faf-aa0f-c2fe51d30380 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1655 microversion: 2.1 time: 0.209681 Jul 24 01:04:28.261608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 249/511] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:28 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1655 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:28.276955 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-bba764c9-e3f4-4f4b-859d-e3f5aad7aeca tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:28.283868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bba764c9-e3f4-4f4b-859d-e3f5aad7aeca tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.284315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bba764c9-e3f4-4f4b-859d-e3f5aad7aeca tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.306309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-bba764c9-e3f4-4f4b-859d-e3f5aad7aeca tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Going to try to start instance {{(pid=117956) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 24 01:04:28.348718 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-bba764c9-e3f4-4f4b-859d-e3f5aad7aeca tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action" status: 202 len: 0 microversion: 2.1 time: 0.074905 Jul 24 01:04:28.349304 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 263/512] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:04:28 2022] POST /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/action => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:04:28.357551 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:28.358320 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:28.365742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.366237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.367150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:28.367597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:28.514409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:28.522691 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f8d6140a-26e9-46be-8132-98f92be05a66 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1664 microversion: 2.1 time: 0.168980 Jul 24 01:04:28.523354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 250/513] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:28 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1664 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:29.278429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:29.279170 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:29.285683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:29.286130 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:29.286915 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:29.287348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:29.423766 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:29.432777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-960741aa-d274-40fb-abbd-1fe53fe1787b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.158464 Jul 24 01:04:29.433364 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 264/514] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:29 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:29.540777 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:29.541414 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:29.548183 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:29.548628 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:29.549641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:29.550085 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:29.677241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:29.684592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c190aa93-0da8-4da2-8e7e-fe3f923c2535 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1664 microversion: 2.1 time: 0.147859 Jul 24 01:04:29.685157 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 251/515] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:29 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1664 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:30.450723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:30.451402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:30.458209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:30.458653 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:30.459433 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:30.459864 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:30.599977 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:30.608031 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e94c34b8-7468-4e99-bd47-0864d3d44be5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.161153 Jul 24 01:04:30.608591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 265/516] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:30 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:30.701669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:30.702275 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:30.709398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:30.709846 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:30.710611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:30.711070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:30.838556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:30.846248 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e3ad5545-2d2d-4500-b801-c134d9596832 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1668 microversion: 2.1 time: 0.148410 Jul 24 01:04:30.847553 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 252/517] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:30 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1668 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:31.625767 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:31.626417 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:31.634295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:31.634742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:31.635526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:31.635955 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:31.767287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:31.775955 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ccb20257-ec75-4115-bb75-71c209ac4cc2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.154143 Jul 24 01:04:31.776635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 266/518] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:31 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:32.793872 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:32.794481 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:32.801733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:32.802182 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:32.802980 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:32.803419 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:32.975912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:32.984991 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-def73353-db85-4602-b64b-85ad7311e66a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.194960 Jul 24 01:04:32.985561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 253/519] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:32 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:33.065920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:33.066600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:33.073906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:33.074476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:33.075211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:33.075653 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:33.265041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:33.273992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0c254776-d3f4-485e-b1f0-1ba50098c858 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1595 microversion: 2.1 time: 0.212311 Jul 24 01:04:33.274645 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 267/520] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:33 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1595 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:34.001448 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:34.002070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:34.004632 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:34.010107 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:34.010572 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:34.011397 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:34.011835 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:34.164599 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:34.173596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d1ef7388-429a-4241-8904-f0c223f47118 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.175815 Jul 24 01:04:34.174285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 254/521] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:33 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:35.192158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:35.192789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:35.199492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:35.199933 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:35.200692 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:35.201130 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:35.308474 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-ecb00c67-278c-4e0d-b0ae-6184ad3d271f req-6597e05f-ccd4-4b96-99ca-e5a84dbecf63 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:35.317849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-ecb00c67-278c-4e0d-b0ae-6184ad3d271f req-6597e05f-ccd4-4b96-99ca-e5a84dbecf63 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:35.318304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-ecb00c67-278c-4e0d-b0ae-6184ad3d271f req-6597e05f-ccd4-4b96-99ca-e5a84dbecf63 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:35.330134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-ecb00c67-278c-4e0d-b0ae-6184ad3d271f req-6597e05f-ccd4-4b96-99ca-e5a84dbecf63 service nova] Creating event network-changed:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:04:35.336196 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-ecb00c67-278c-4e0d-b0ae-6184ad3d271f req-6597e05f-ccd4-4b96-99ca-e5a84dbecf63 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031119 Jul 24 01:04:35.339730 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 255/522] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:04:35.386273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:35.400099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0b2445a4-0d35-49aa-9ddd-137930e40642 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.212965 Jul 24 01:04:35.402702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 268/523] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:35 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:36.418447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:36.422575 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:36.430992 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:36.431763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:36.432625 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:36.432975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:36.585731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:36.594091 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-df75c5d9-4d14-483d-9c2e-09488d0bd76c tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.180186 Jul 24 01:04:36.594654 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 256/524] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:36 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:36.594896 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1372850725"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:36.596279 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:36.598947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=84 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:36.604852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:36.605296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:36.606055 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:36.606486 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:36.738132 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.network.security_group_api [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Adding security group b8b34d0f-b8f7-4710-9499-8209c663a9ff to port 83a88c5b-6010-4bfd-9dd1-ee05588aab25 Jul 24 01:04:37.090565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e3cea3c1-5c31-4185-8450-04e5595cd9b3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/action" status: 202 len: 0 microversion: 2.1 time: 0.500055 Jul 24 01:04:37.091281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 269/525] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:04:36 2022] POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/action => generated 0 bytes in 501 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:04:37.100025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-b6f95e8d-2727-4f5b-bffd-7cb6ba310966 req-0e137c10-00af-4bac-9d06-9b37d8d4ad15 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:37.100784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:37.101470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:37.108140 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-b6f95e8d-2727-4f5b-bffd-7cb6ba310966 req-0e137c10-00af-4bac-9d06-9b37d8d4ad15 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:37.108823 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-b6f95e8d-2727-4f5b-bffd-7cb6ba310966 req-0e137c10-00af-4bac-9d06-9b37d8d4ad15 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:37.110684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:37.111219 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:37.112577 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:37.113023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:37.131519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-b6f95e8d-2727-4f5b-bffd-7cb6ba310966 req-0e137c10-00af-4bac-9d06-9b37d8d4ad15 service nova] Creating event network-changed:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:04:37.136708 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-b6f95e8d-2727-4f5b-bffd-7cb6ba310966 req-0e137c10-00af-4bac-9d06-9b37d8d4ad15 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041591 Jul 24 01:04:37.137329 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 257/526] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:37.264887 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:37.273545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-242ac374-19c5-4faf-9f3e-7b79ee024cbe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1761 microversion: 2.1 time: 0.178339 Jul 24 01:04:37.274106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 270/527] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:37 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1761 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:37.612615 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:37.613325 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:37.619596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:37.620055 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:37.620827 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:37.621271 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:37.638171 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:37.756755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:37.765165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e6565c6f-a659-4833-bfb3-5a3820066f2f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.157948 Jul 24 01:04:37.765736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 258/528] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:37 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:38.783720 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:38.784480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:38.784598 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:38.785078 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:38.787403 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=78 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:38.791060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.791513 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.792285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.792865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.792968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.793393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.794160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.794597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.818610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bd6a7d9e-89dc-473c-930b-57b8582d616c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/ips" status: 200 len: 151 microversion: 2.1 time: 0.037840 Jul 24 01:04:38.819462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 259/529] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:04:38 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/ips => generated 151 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:04:38.830385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "af294a16-ac68-4fe3-b3fb-db77a673a732"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:38.831565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:38.837715 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.838169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.839050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.839488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.927499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Invalid volume: volume af294a16-ac68-4fe3-b3fb-db77a673a732 is already attached to instances: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: POST /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/os-volume_attachments HTTP/1.1 Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 74 Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:04:38.929529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {"volumeAttachment": {"volumeId": "af294a16-ac68-4fe3-b3fb-db77a673a732"}} Jul 24 01:04:38.931149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Invalid volume: volume af294a16-ac68-4fe3-b3fb-db77a673a732 is already attached to instances: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'server_id': 'fd889446-7a8d-4e52-91a5-22b75a923a21', 'body': {'volumeAttachment': {'volumeId': 'af294a16-ac68-4fe3-b3fb-db77a673a732'}}} Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume af294a16-ac68-4fe3-b3fb-db77a673a732 is already attached to instances: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:38.932069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume af294a16-ac68-4fe3-b3fb-db77a673a732 is already attached to instances: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:38.933440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:38.934877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:38.934877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:04:38.934877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:04:38.934877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume af294a16-ac68-4fe3-b3fb-db77a673a732 is already attached to instances: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a Jul 24 01:04:38.934877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:38.936081 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 400 to user: Invalid volume: volume af294a16-ac68-4fe3-b3fb-db77a673a732 is already attached to instances: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:04:38.936947 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1eb3d880-4657-4cad-819c-04f6da41bc44 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.110385 Jul 24 01:04:38.937541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 260/530] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:04:38 2022] POST /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21/os-volume_attachments => generated 174 bytes in 111 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 24 01:04:38.942217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:38.946692 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:38.947417 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:38.953540 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.954190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.954923 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-17c5da6e-6283-4140-96c4-a7bc8bc09b86 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.175029 Jul 24 01:04:38.955316 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:38.956433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:38.956808 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 271/531] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:38 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:38.983652 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:04:39.086746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-061c6c9f-169a-459d-b220-c9b9f45eb361 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 204 len: 0 microversion: 2.1 time: 0.143091 Jul 24 01:04:39.087505 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 261/532] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:04:38 2022] DELETE /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 24 01:04:39.096824 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:39.097505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:39.103676 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:39.104133 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:39.104911 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:39.105339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:39.236067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:39.244398 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-336232f1-8b74-4da3-88b7-fdbd9cfa73df tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1684 microversion: 2.1 time: 0.152246 Jul 24 01:04:39.244976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 272/533] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:39 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1684 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:39.973753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:39.974445 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:39.980887 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:39.981330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:39.982083 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:39.982513 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:40.113750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:40.122219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9eb1e9af-a960-43ec-a7f0-156c74089b14 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.153355 Jul 24 01:04:40.122978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 262/534] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:39 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:40.261468 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:40.262087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:40.269272 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:40.269725 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:40.270497 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:40.270975 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:40.396938 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:40.404391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0d93ebc0-ad11-4e52-989e-c42b818eab9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1684 microversion: 2.1 time: 0.146859 Jul 24 01:04:40.405026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 273/535] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:40 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1684 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:41.139734 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:41.140343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:41.147397 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:41.147850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:41.148613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:41.149050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:41.286168 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:41.306889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7b9124b4-09ba-4da9-a14b-1f87e64209f0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.170604 Jul 24 01:04:41.324755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 263/536] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:41 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:41.422683 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:41.423341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:41.430348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:41.430729 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:41.431524 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:41.431957 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:41.639211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:41.647611 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-516dc6a5-7cdb-4bff-9ebb-6d7f7336d0f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1684 microversion: 2.1 time: 0.228883 Jul 24 01:04:41.648299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 274/537] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:41 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1684 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:42.280169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-bd219152-ca75-46e8-8ab4-fd3d865243c7 req-403a22cf-d633-4572-840b-6324b05f4700 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd889446-7a8d-4e52-91a5-22b75a923a21", "name": "network-vif-deleted", "tag": "0a201010-559e-40e9-8371-e03c2a2e2d4c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:42.290741 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-bd219152-ca75-46e8-8ab4-fd3d865243c7 req-403a22cf-d633-4572-840b-6324b05f4700 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:42.291223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-bd219152-ca75-46e8-8ab4-fd3d865243c7 req-403a22cf-d633-4572-840b-6324b05f4700 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:42.304307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-bd219152-ca75-46e8-8ab4-fd3d865243c7 req-403a22cf-d633-4572-840b-6324b05f4700 service nova] Creating event network-vif-deleted:0a201010-559e-40e9-8371-e03c2a2e2d4c for instance fd889446-7a8d-4e52-91a5-22b75a923a21 on hv-cinder-maste Jul 24 01:04:42.309300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-bd219152-ca75-46e8-8ab4-fd3d865243c7 req-403a22cf-d633-4572-840b-6324b05f4700 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032949 Jul 24 01:04:42.310206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 264/538] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:42.340644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:42.341248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:42.348537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:42.348988 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:42.350931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:42.351548 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:42.355793 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:42.526463 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:42.536954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4d935bbc-3fc4-4081-9b4b-e5241d4bdc8b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.200191 Jul 24 01:04:42.547422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 275/539] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:42 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:42.667636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:42.668679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:42.677332 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:42.677890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:42.678740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:42.679288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:42.713832 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:42.714642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:42.715331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lazy-loading 'fault' on Instance uuid fd889446-7a8d-4e52-91a5-22b75a923a21 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:04:42.763502 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:42.772272 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a1de1348-7e39-44b1-bff3-2f08692217c1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 200 len: 1332 microversion: 2.1 time: 0.109620 Jul 24 01:04:42.772790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 265/540] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:42 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 1332 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:43.554810 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:43.556108 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:43.567186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:43.567705 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:43.570515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:43.572021 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:43.718607 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:43.728024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-dc94bb60-d1ad-45ec-9cfd-c4e91fc7c154 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.177846 Jul 24 01:04:43.728669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 276/541] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:43 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:43.789711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:43.790259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:43.797078 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:43.797531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:43.798305 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:43.798739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:43.813523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 HTTP/1.1 Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:04:43.816025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:04:43.817433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'fd889446-7a8d-4e52-91a5-22b75a923a21'} Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:04:43.818139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:04:43.820638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:43.821933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:04:43.821933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d3c29c7f-ed2b-41c3-b3d5-848182bd3bb2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 404 len: 111 microversion: 2.1 time: 0.031867 Jul 24 01:04:43.821933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 266/542] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:43 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:04:43.826608 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-89eb5522-27ce-4a29-bed2-cdc2997f021b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '.version_select at 0x7f5e304a2af0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:43.847349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-89eb5522-27ce-4a29-bed2-cdc2997f021b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1714377476" status: 202 len: 0 microversion: 2.1 time: 0.023428 Jul 24 01:04:43.851496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 277/543] 10.222.0.18 () {58 vars in 1220 bytes} [Sun Jul 24 01:04:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1714377476 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:04:44.498845 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:44.499337 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:44.505945 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:44.506396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:44.507186 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:44.507621 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:44.587582 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lazy-loading 'services' on Instance uuid 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:04:44.658116 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-10d7934e-f44d-4b76-a2c2-d96d7859d2b0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments/af294a16-ac68-4fe3-b3fb-db77a673a732" status: 202 len: 0 microversion: 2.1 time: 0.163051 Jul 24 01:04:44.658826 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 267/544] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:04:44 2022] DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments/af294a16-ac68-4fe3-b3fb-db77a673a732 => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:04:44.667454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:44.668364 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:44.676162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:44.676418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:44.677571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:44.678051 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:44.712365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-bda0acaa-ff0f-47c9-9a33-088bc15434f5 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048851 Jul 24 01:04:44.713095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 278/545] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:04:44 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:44.746098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:44.746955 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:44.753906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:44.754366 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:44.755153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:44.755594 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:44.891148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:44.899192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c859ed5d-9921-4c1e-814e-4a47321b035f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.157602 Jul 24 01:04:44.899759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 268/546] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:44 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:45.723938 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:45.724850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:45.731284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:45.731732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:45.732492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:45.732946 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:45.762617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0f2dbec9-b010-4454-9ff9-08398122f105 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042682 Jul 24 01:04:45.763273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 279/547] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:04:45 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:45.915849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:45.916478 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:45.923042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:45.923488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:45.924929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:45.925636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:46.052465 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:46.060531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2193d780-9931-483f-bd12-c3685a4f141f tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.148527 Jul 24 01:04:46.061105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 269/548] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:45 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:46.775015 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:46.775891 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:46.782375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:46.782851 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:46.783745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:46.784370 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:46.812642 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-29278188-9cc1-4138-8f16-34ae3c32f0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041612 Jul 24 01:04:46.813188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 280/549] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:04:46 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:04:46.869563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:46.870062 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:46.876214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:46.876667 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:46.877996 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:46.878435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:46.903205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:04:46.977461 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5286cad2-95c1-4901-99a6-dca776f81d0b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 204 len: 0 microversion: 2.1 time: 0.111521 Jul 24 01:04:46.978510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 270/550] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:04:46 2022] DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:04:46.986706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:46.987474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:46.993461 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:46.993913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:46.994682 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:46.995144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:47.077973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:47.078728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:47.085192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:47.085642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:47.086461 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:47.086925 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:47.138458 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:47.146677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a92f7d67-825a-45f9-b78a-e737db264fce tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1682 microversion: 2.1 time: 0.164176 Jul 24 01:04:47.147361 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 281/551] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:46 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1682 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:47.220038 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:47.228711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ed4abbf1-2c54-406f-8962-d56c4ddf3f31 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.155018 Jul 24 01:04:47.229262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 271/552] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:47 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:48.164012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:48.164613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:48.171082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:48.171530 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:48.172279 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:48.172710 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:48.245990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:48.246606 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:48.253365 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:48.253817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:48.254595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:48.255073 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:48.271870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:48.313604 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:48.321760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-79d51575-1690-4beb-8326-61e14165eb10 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1682 microversion: 2.1 time: 0.161522 Jul 24 01:04:48.322347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 282/553] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:48 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:48.636692 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:48.645355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d9944679-7293-4473-8c5f-2f31f19bf310 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.403703 Jul 24 01:04:48.645919 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 272/554] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:48 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:49.339533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:49.340274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:49.347062 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:49.347510 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:49.348281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:49.348717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:49.477939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:49.485481 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b3a4f000-995b-48e1-b0ef-3997850b6e7f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1682 microversion: 2.1 time: 0.149967 Jul 24 01:04:49.486050 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 283/555] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:49 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1682 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:49.661931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:49.662639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:49.669143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:49.669601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:49.670373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:49.670831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:49.804521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:49.812498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ffcbd9d7-859e-480f-a799-3ce393dcce49 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.154615 Jul 24 01:04:49.813066 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 273/556] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:49 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:49.948459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-94d11b83-9066-487f-a369-4c5e408c379c req-4c615f50-a1f9-459f-8838-77461dadbebb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c006a24-d0c5-4ab6-9ce4-259e05a9c56a", "name": "network-vif-deleted", "tag": "3e10a0f3-72ea-499e-9aa7-f84921595c2e"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:49.956275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-94d11b83-9066-487f-a369-4c5e408c379c req-4c615f50-a1f9-459f-8838-77461dadbebb service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:49.956719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-94d11b83-9066-487f-a369-4c5e408c379c req-4c615f50-a1f9-459f-8838-77461dadbebb service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:49.967438 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-94d11b83-9066-487f-a369-4c5e408c379c req-4c615f50-a1f9-459f-8838-77461dadbebb service nova] Creating event network-vif-deleted:3e10a0f3-72ea-499e-9aa7-f84921595c2e for instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a on hv-cinder-maste Jul 24 01:04:49.973472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-94d11b83-9066-487f-a369-4c5e408c379c req-4c615f50-a1f9-459f-8838-77461dadbebb service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028694 Jul 24 01:04:49.974055 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 284/557] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:50.503647 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:50.504255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:50.510420 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:50.511285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:50.511690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:50.512128 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:50.589608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:50.599407 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d0dd7967-72f8-4e33-be22-9f6827f4b173 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1326 microversion: 2.1 time: 0.099738 Jul 24 01:04:50.600593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 274/558] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:50 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1326 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:50.831126 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:50.831741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:50.838217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:50.838665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:50.839474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:50.839911 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:50.975421 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:50.983438 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c4611abc-5746-4caa-98bc-27e28d23a16e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1732 microversion: 2.1 time: 0.157701 Jul 24 01:04:50.984011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 285/559] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:50 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1732 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:51.616960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:51.617565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:51.624276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:51.624716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:51.625469 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:51.625897 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:51.660644 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:51.661147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:51.661594 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lazy-loading 'fault' on Instance uuid 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:04:51.708061 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:51.715859 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4b8bdbcb-8ed4-4fd5-82b2-f9f633e0ea39 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 200 len: 1331 microversion: 2.1 time: 0.102781 Jul 24 01:04:51.716457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 275/560] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:51 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 1331 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:51.999937 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:52.000714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:52.007234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.007675 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.008529 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.008991 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.135217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:52.143098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f7f68a2b-0e02-4f4a-a71a-4848befcc270 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1712 microversion: 2.1 time: 0.147100 Jul 24 01:04:52.143697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 286/561] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:51 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1712 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:52.244819 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:52.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:52.251623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.252018 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.252870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.253310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.336544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] HTTP exception thrown: Cannot 'detach_volume' instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e while it is in vm_state rescued Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Request method failure captured: Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments/d1a3d8e7-19a4-40bf-a311-17e785b4bf50 HTTP/1.1 Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnFiejBFZa8mJ5D502lXi0vI2AJZSXzLQ6yvLVNBpg5wuvRciV8GsNNEfmF5OiwQV_hyTlbh1vvTV_RBBoRfFvSvMn5HPevJqiFZ0JTvv9BtvBk2xvVAY9_4RTM-tU7gCIt6rWRh-Ok03kE5HtQWgevEOi1XOD-L9IgzsF_0g4DZ8U Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:04:52.338138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_f7d219e82efe44f9b6f860a5fabc7733", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 283a1bfe0e3b45aaa0a33e5fe6b1d85a Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:04:52.339744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Cannot 'detach_volume' instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e while it is in vm_state rescued Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'server_id': 'c6a71fa6-32c2-446f-b9e5-80d4e9ea800e', 'id': 'd1a3d8e7-19a4-40bf-a311-17e785b4bf50'} Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:04:52.340624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e while it is in vm_state rescued Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e while it is in vm_state rescued Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.342121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Returning 409 to user: Cannot 'detach_volume' instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e while it is in vm_state rescued {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:04:52.343284 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4ad2354d-46a6-41df-9c95-5c08d352b3b8 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments/d1a3d8e7-19a4-40bf-a311-17e785b4bf50" status: 409 len: 152 microversion: 2.1 time: 0.099332 Jul 24 01:04:52.343284 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 276/562] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:04:52 2022] DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments/d1a3d8e7-19a4-40bf-a311-17e785b4bf50 => generated 152 bytes in 100 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 24 01:04:52.349181 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:52.349753 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:52.356007 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.356459 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.357206 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.357636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.423406 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-25122e81-1aa2-4656-be78-4616fb92fa7d tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "POST /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/action" status: 202 len: 0 microversion: 2.1 time: 0.078530 Jul 24 01:04:52.424537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 287/563] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:04:52 2022] POST /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/action => generated 0 bytes in 80 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:04:52.432398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:52.433008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:52.439070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.439545 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.440310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.440749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.570792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:52.578682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1e1ccf07-16e2-4b8b-9cfb-c43614e398f6 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1720 microversion: 2.1 time: 0.150103 Jul 24 01:04:52.579458 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 277/564] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:52 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:52.735078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:52.735911 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:52.743496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.743967 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.744752 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:52.745186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:52.759987 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a HTTP/1.1 Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:04:52.761617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:04:52.763142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '1c006a24-d0c5-4ab6-9ce4-259e05a9c56a'} Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:04:52.764049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:04:52.765426 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:04:52.766918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:04:52.766918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b835f699-8980-4832-98fe-453949cbe039 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 404 len: 111 microversion: 2.1 time: 0.033413 Jul 24 01:04:52.766918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 288/565] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:52 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:04:52.773012 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ff552dcc-2021-4613-9780-c0671fe6172b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '.version_select at 0x7f5e304da5e0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:52.796503 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ff552dcc-2021-4613-9780-c0671fe6172b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2127023780" status: 202 len: 0 microversion: 2.1 time: 0.027246 Jul 24 01:04:52.797073 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 278/566] 10.222.0.18 () {58 vars in 1220 bytes} [Sun Jul 24 01:04:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2127023780 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:04:53.413434 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ba964b01-ac74-4904-a24b-fb8eda05a915 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: .version_select at 0x7f5e3045d700>, body: {"keypair": {"name": "tempest-keypair-1444987007", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO2RqOGsMkw4Uf7NQisPzZDFiH5oMPtITLUPRV8ItgAD7lXMuPyrI0qwxeh685gv9yE1S0oXR6E3K+wqURoNX1wzeB6p8Dsm0ybaNJKcqIrfpaA4/c7OJHejQ4PSbOsQDg=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:53.429195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-ba964b01-ac74-4904-a24b-fb8eda05a915 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:04:53.433447 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-ba964b01-ac74-4904-a24b-fb8eda05a915 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:04:53.456845 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ba964b01-ac74-4904-a24b-fb8eda05a915 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047509 Jul 24 01:04:53.457502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 289/567] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:04:53 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:04:53.597631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:53.598237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:53.604940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:53.605314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:53.606077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:53.606791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:53.800906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:53.809275 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1648d583-aab4-4231-a17f-7b062294e99b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1720 microversion: 2.1 time: 0.216359 Jul 24 01:04:53.809865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 279/568] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:53 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1720 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:54.745030 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2019010703", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "37cd1572-14d0-471e-b2ca-ff3f9078882e"}], "security_groups": [{"name": "tempest-securitygroup--256220103"}], "key_name": "tempest-keypair-1444987007", "user_data": "}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:54.827094 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:54.827706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:54.834646 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:54.835135 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:54.835899 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:54.836334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:54.845640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] validate_networks() for [('37cd1572-14d0-471e-b2ca-ff3f9078882e', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:04:55.043931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:55.052728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4bfd1dc8-0ce6-4910-9423-d873907e1ffc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1720 microversion: 2.1 time: 0.229490 Jul 24 01:04:55.053393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 280/569] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:54 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1720 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:04:55.280209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:04:55.280634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:04:55.281106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:04:55.281513 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:04:55.281922 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:04:55.291515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:55.291965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:55.292761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:55.293195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:55.310165 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:04:55.314117 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:04:55.325685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:04:55.327964 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:04:55.373388 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5b2766ed-c3d3-4b64-bd72-14845ac3d841 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.633302 Jul 24 01:04:55.374029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 290/570] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:04:54 2022] POST /compute/v2.1/servers => generated 401 bytes in 635 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:04:55.384232 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ff79727b-5440-4c09-93df-ac3e93d4a6d4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:55.384838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ff79727b-5440-4c09-93df-ac3e93d4a6d4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:55.437898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ff79727b-5440-4c09-93df-ac3e93d4a6d4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:55.445344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ff79727b-5440-4c09-93df-ac3e93d4a6d4 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1242 microversion: 2.1 time: 0.065080 Jul 24 01:04:55.445912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 281/571] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:55 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1242 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:56.070865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:56.071508 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:56.078857 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:56.079348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:56.080274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:56.080792 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:56.208151 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:56.216212 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6ab971e1-24e0-44f2-a0a3-b17668e20203 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1726 microversion: 2.1 time: 0.149690 Jul 24 01:04:56.216774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 291/572] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:56 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1726 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:56.232544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:56.233022 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:56.240185 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:56.240574 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:56.241327 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:56.241762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:56.319949 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5e856273-2709-4b3c-9860-9ed97bb660cc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091149 Jul 24 01:04:56.320539 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 282/573] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:04:56 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:04:56.461553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:56.462418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:56.468664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:56.469113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:56.469878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:56.470311 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:56.572288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:56.580251 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e94a7191-b052-41d6-b909-386fd9d4d30e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1294 microversion: 2.1 time: 0.122547 Jul 24 01:04:56.581019 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 292/574] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:56 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1294 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:57.257858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-b830e4bf-dd9f-47bc-8e27-21e35f2ddfe0 req-948b1e1e-e43f-4d8f-bb8e-9b7d06d9d351 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6a71fa6-32c2-446f-b9e5-80d4e9ea800e", "tag": "c3604b8e-6ace-40d2-8c7a-9f222b26d78c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:57.269775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-b830e4bf-dd9f-47bc-8e27-21e35f2ddfe0 req-948b1e1e-e43f-4d8f-bb8e-9b7d06d9d351 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:57.270210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-b830e4bf-dd9f-47bc-8e27-21e35f2ddfe0 req-948b1e1e-e43f-4d8f-bb8e-9b7d06d9d351 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:57.281911 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-b830e4bf-dd9f-47bc-8e27-21e35f2ddfe0 req-948b1e1e-e43f-4d8f-bb8e-9b7d06d9d351 service nova] Creating event network-changed:c3604b8e-6ace-40d2-8c7a-9f222b26d78c for instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e on hv-cinder-maste Jul 24 01:04:57.287408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-b830e4bf-dd9f-47bc-8e27-21e35f2ddfe0 req-948b1e1e-e43f-4d8f-bb8e-9b7d06d9d351 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033880 Jul 24 01:04:57.288396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 283/575] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:57.598960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:57.599583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:57.610298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:57.610803 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:57.613041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:57.613488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:58.140787 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:58.149963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-673a23c6-d1d1-4f97-9b6d-647393d64a4b tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.556226 Jul 24 01:04:58.151283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 293/576] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:57 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:58.247233 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-7bc464af-ddaf-44bd-a713-ba5aff939409 req-dd8e99dd-3a3e-4b1a-b488-d8f8d3eb78f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4693fbdb-0311-4f58-a0b7-e338e3369224", "tag": "6545a60f-fbf2-432b-a011-f05ea4e793a1"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:58.256931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-7bc464af-ddaf-44bd-a713-ba5aff939409 req-dd8e99dd-3a3e-4b1a-b488-d8f8d3eb78f7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:58.257314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-7bc464af-ddaf-44bd-a713-ba5aff939409 req-dd8e99dd-3a3e-4b1a-b488-d8f8d3eb78f7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:58.290357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-7bc464af-ddaf-44bd-a713-ba5aff939409 req-dd8e99dd-3a3e-4b1a-b488-d8f8d3eb78f7 service nova] Creating event network-changed:6545a60f-fbf2-432b-a011-f05ea4e793a1 for instance 4693fbdb-0311-4f58-a0b7-e338e3369224 on hv-cinder-maste Jul 24 01:04:58.304962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-7bc464af-ddaf-44bd-a713-ba5aff939409 req-dd8e99dd-3a3e-4b1a-b488-d8f8d3eb78f7 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061894 Jul 24 01:04:58.306311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 284/577] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:04:59.167204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:04:59.167814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:04:59.174404 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:59.175152 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:59.176151 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:59.176588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:59.303043 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-216d1f43-5481-47df-9503-3c7e7b5c08d1 req-5e19323f-a1e1-45b9-af4e-d1f9e6c008c6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6a71fa6-32c2-446f-b9e5-80d4e9ea800e", "tag": "c3604b8e-6ace-40d2-8c7a-9f222b26d78c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:04:59.311116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:04:59.311259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-216d1f43-5481-47df-9503-3c7e7b5c08d1 req-5e19323f-a1e1-45b9-af4e-d1f9e6c008c6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:04:59.311618 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-216d1f43-5481-47df-9503-3c7e7b5c08d1 req-5e19323f-a1e1-45b9-af4e-d1f9e6c008c6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:04:59.313598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:04:59.321228 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fff73198-7d44-4b34-a479-03bb0c721374 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.157931 Jul 24 01:04:59.321837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 294/578] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:04:59 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:04:59.323035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-216d1f43-5481-47df-9503-3c7e7b5c08d1 req-5e19323f-a1e1-45b9-af4e-d1f9e6c008c6 service nova] Creating event network-changed:c3604b8e-6ace-40d2-8c7a-9f222b26d78c for instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e on hv-cinder-maste Jul 24 01:04:59.327728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-216d1f43-5481-47df-9503-3c7e7b5c08d1 req-5e19323f-a1e1-45b9-af4e-d1f9e6c008c6 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029403 Jul 24 01:04:59.328306 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 285/579] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:04:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:05:00.339488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:00.340099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:00.348355 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:00.348880 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:00.349671 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:00.350110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:00.531142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:00.539162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e20cb68b-31c3-43aa-9406-4bee7436a6a2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.203843 Jul 24 01:05:00.539729 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 295/580] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:00 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:00.848516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:00.849030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:00.855418 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:00.855874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:00.856664 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:00.857105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:00.886623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d83ca8f9-97fb-4e2b-8c6a-abb80917522b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments/d78a3415-3fd7-4061-b9c5-cc2f467e75e0" status: 200 len: 194 microversion: 2.20 time: 0.043171 Jul 24 01:05:00.887259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 286/581] 10.222.0.18 () {60 vars in 1453 bytes} [Sun Jul 24 01:05:00 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments/d78a3415-3fd7-4061-b9c5-cc2f467e75e0 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:05:00.949722 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:00.950214 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:00.956296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:00.956747 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:00.957515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:00.957949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:01.039432 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lazy-loading 'services' on Instance uuid 281e8695-c8e7-4293-8abe-a4f88f175084 {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:05:01.115589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-629c842c-6c9f-4460-8978-8ea75d193d38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments/d78a3415-3fd7-4061-b9c5-cc2f467e75e0" status: 202 len: 0 microversion: 2.20 time: 0.169624 Jul 24 01:05:01.116196 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 296/582] 10.222.0.18 () {60 vars in 1456 bytes} [Sun Jul 24 01:05:00 2022] DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments/d78a3415-3fd7-4061-b9c5-cc2f467e75e0 => generated 0 bytes in 171 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:05:01.124748 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:01.125921 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:01.132678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:01.133140 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:01.133922 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:01.134392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:01.168211 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0b69b75a-8f5c-49bc-9873-b074ea126604 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047501 Jul 24 01:05:01.168782 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 287/583] 10.222.0.18 () {60 vars in 1342 bytes} [Sun Jul 24 01:05:01 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:05:01.556871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:01.557586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:01.563780 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:01.564282 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:01.565073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:01.565504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:01.697717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:01.706618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-85cbda39-7b84-4caf-9432-0b20bf122b8a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.154386 Jul 24 01:05:01.707504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 297/584] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:01 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:02.180968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:02.181953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:02.191878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.192502 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.193492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.194180 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.225056 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-dfcc7573-fa96-4b8f-b7ec-232f47911b1c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.048074 Jul 24 01:05:02.225607 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 288/585] 10.222.0.18 () {60 vars in 1342 bytes} [Sun Jul 24 01:05:02 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:02.285669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:02.286156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:02.292159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.292596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.293348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.293776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.320086 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:05:02.396788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2c807901-4b9e-4b54-82a3-a1099b05bac9 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 204 len: 0 microversion: 2.20 time: 0.115362 Jul 24 01:05:02.397599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 298/586] 10.222.0.18 () {60 vars in 1279 bytes} [Sun Jul 24 01:05:02 2022] DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 24 01:05:02.406209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:02.406836 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:02.412951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.413405 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.414174 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.414611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.542645 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:02.551236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:02.552746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:02.553902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-43e8c7d8-d2a2-43a5-a60c-6016495d0277 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.151893 Jul 24 01:05:02.554456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 289/587] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:02 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:05:02.724999 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:02.725694 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:02.732587 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.733033 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.733796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:02.734326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:02.865651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:02.873015 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fd9d9391-8b28-4f9d-86ea-dc2e32405c98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.153080 Jul 24 01:05:02.873572 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 299/588] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:02 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:03.088301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-f8e68a21-afff-4538-bb19-f7059a0a91de req-261c14ec-e033-4333-a003-e5e04e806f5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4693fbdb-0311-4f58-a0b7-e338e3369224", "name": "network-vif-plugged", "status": "completed", "tag": "6545a60f-fbf2-432b-a011-f05ea4e793a1"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:03.096568 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-f8e68a21-afff-4538-bb19-f7059a0a91de req-261c14ec-e033-4333-a003-e5e04e806f5d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:03.097212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-f8e68a21-afff-4538-bb19-f7059a0a91de req-261c14ec-e033-4333-a003-e5e04e806f5d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:03.108912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-f8e68a21-afff-4538-bb19-f7059a0a91de req-261c14ec-e033-4333-a003-e5e04e806f5d service nova] Creating event network-vif-plugged:6545a60f-fbf2-432b-a011-f05ea4e793a1 for instance 4693fbdb-0311-4f58-a0b7-e338e3369224 on hv-cinder-maste Jul 24 01:05:03.114500 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-f8e68a21-afff-4538-bb19-f7059a0a91de req-261c14ec-e033-4333-a003-e5e04e806f5d service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029914 Jul 24 01:05:03.115125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 290/589] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:03.572739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:03.573342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:03.579376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:03.579831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:03.581083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:03.581523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:03.715023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:03.723945 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:03.725470 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:03.726648 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8238a6cb-1c80-4aad-b52e-fae0be17cbc5 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.158100 Jul 24 01:05:03.727237 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 300/590] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:03 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:03.890045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:03.890736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:03.897612 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:03.898058 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:03.898887 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:03.899325 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:04.032417 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:04.039948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4401eb7a-b3c6-4241-bfaa-2e126a8b40e2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.154575 Jul 24 01:05:04.040519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 291/591] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:03 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:04.745234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:04.746032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:04.753186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:04.753640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:04.754431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:04.754888 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:04.943228 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:04.951346 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:04.952837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:04.953986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-be15b7b1-2bb0-46ab-8fa9-35a08222e64b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1722 microversion: 2.20 time: 0.212726 Jul 24 01:05:04.954540 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 301/592] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:04 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1722 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:05.056589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:05.057195 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:05.063840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:05.064283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:05.065072 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:05.065596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:05.271505 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:05.278933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0d97d607-fd84-4330-b1bf-1cef1b4d67f2 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1367 microversion: 2.1 time: 0.226193 Jul 24 01:05:05.279614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 292/593] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:05 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1367 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:05.385018 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-2a8b740f-3827-495a-9fd4-3f670e28eb34 req-4174a8fe-0145-4918-844a-ee0baa27928a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "281e8695-c8e7-4293-8abe-a4f88f175084", "name": "network-vif-deleted", "tag": "e11dcec4-66a1-456b-85bc-587d87bc9e6a"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:05.388783 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:05.389526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=10 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:05.390417 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=1,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:05.404111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2a8b740f-3827-495a-9fd4-3f670e28eb34 req-4174a8fe-0145-4918-844a-ee0baa27928a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:05.404565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2a8b740f-3827-495a-9fd4-3f670e28eb34 req-4174a8fe-0145-4918-844a-ee0baa27928a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:05.427178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-2a8b740f-3827-495a-9fd4-3f670e28eb34 req-4174a8fe-0145-4918-844a-ee0baa27928a service nova] Creating event network-vif-deleted:e11dcec4-66a1-456b-85bc-587d87bc9e6a for instance 281e8695-c8e7-4293-8abe-a4f88f175084 on hv-cinder-maste Jul 24 01:05:05.432772 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-2a8b740f-3827-495a-9fd4-3f670e28eb34 req-4174a8fe-0145-4918-844a-ee0baa27928a service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051292 Jul 24 01:05:05.433746 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 302/594] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:05.974704 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:05.975497 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:05.977502 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:05.983143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:05.983659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:05.984433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:05.984861 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:06.015665 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:06.016113 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:06.016563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lazy-loading 'fault' on Instance uuid 281e8695-c8e7-4293-8abe-a4f88f175084 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:05:06.020063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=100 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:06.059965 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:06.069535 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:06.071105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:06.072266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c378f918-b67a-4c14-869f-a5bcba2fdb6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 200 len: 1371 microversion: 2.20 time: 0.102067 Jul 24 01:05:06.072858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 293/595] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:05 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 1371 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:06.297090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:06.297865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:06.306165 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:06.306742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:06.307690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:06.308185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:06.440526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:06.448329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-61753d1c-65e4-42ff-8b9d-8839209361b6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1557 microversion: 2.1 time: 0.155625 Jul 24 01:05:06.448920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 303/596] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:06 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1557 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:06.465240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:06.465731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:06.472444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:06.472900 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:06.473675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:06.474116 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:06.548140 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b81133db-df73-48a5-a046-389366b131e9 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.086690 Jul 24 01:05:06.548684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 294/597] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:05:06 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-interface => generated 284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:07.091290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:07.092043 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:07.099085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:07.099543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:07.100320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:07.100771 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:07.114998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] HTTP exception thrown: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Request method failure captured: Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 HTTP/1.1 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm9HYDgM9sKkcXoQzs8hJi4Lhvo-EPUoQwkjM0J25ggK_d-5LQx1tfWsBOgFVQxk8NaQ2JPzC3inqUcK2MCKp5-NaPPa10p1dI04lzI-IULSiFk043Gi5E6XOGZFE9jnTv8QQOR0z11fRq7Klx1D_JA_dbRl6IcJI-0eRqM7IltuAk Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Openstack-Nova-Api-Version: 2.20 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:05:07.116699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75293bf0100f435bb567b690edc85533"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75293bf0100f435bb567b690edc85533", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 9a9256b6d7e24c54b061673cd17217cd Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:05:07.118381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '281e8695-c8e7-4293-8abe-a4f88f175084'} Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:05:07.119489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:05:07.121004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:07.122813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Returning 404 to user: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:05:07.122813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-07b756d6-b419-4bef-bc0d-c05849da66d8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 404 len: 111 microversion: 2.20 time: 0.030720 Jul 24 01:05:07.122813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 304/598] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:07 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 24 01:05:07.128295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-99822a9d-09e8-4a58-a48b-b56c630d2c87 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '.version_select at 0x7f5e303ad4c0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:07.145732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-99822a9d-09e8-4a58-a48b-b56c630d2c87 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-424227189" status: 204 len: 0 microversion: 2.20 time: 0.022350 Jul 24 01:05:07.146322 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 295/599] 10.222.0.18 () {60 vars in 1258 bytes} [Sun Jul 24 01:05:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-424227189 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 24 01:05:07.495567 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-526f71ec-4b6e-4960-add7-2c18ef970388 req-ee8b124f-34c7-4d6a-bd81-fa96003d2501 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4693fbdb-0311-4f58-a0b7-e338e3369224", "tag": "6545a60f-fbf2-432b-a011-f05ea4e793a1"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:07.504267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-526f71ec-4b6e-4960-add7-2c18ef970388 req-ee8b124f-34c7-4d6a-bd81-fa96003d2501 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:07.504717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-526f71ec-4b6e-4960-add7-2c18ef970388 req-ee8b124f-34c7-4d6a-bd81-fa96003d2501 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:07.516402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-526f71ec-4b6e-4960-add7-2c18ef970388 req-ee8b124f-34c7-4d6a-bd81-fa96003d2501 service nova] Creating event network-changed:6545a60f-fbf2-432b-a011-f05ea4e793a1 for instance 4693fbdb-0311-4f58-a0b7-e338e3369224 on hv-cinder-maste Jul 24 01:05:07.522631 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-526f71ec-4b6e-4960-add7-2c18ef970388 req-ee8b124f-34c7-4d6a-bd81-fa96003d2501 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030729 Jul 24 01:05:07.524059 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 305/600] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:07.820517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c24242e5-0ec9-41a5-befd-6b5513747040 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'create', calling method: .version_select at 0x7f5e3051c3a0>, body: {"keypair": {"name": "tempest-keypair-274235092", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOXOQm4K3PUgot4VpNG7vTW8zxJp1M7bk3xYH0Z0QEo9pI30XJ3LUfxjeGwevufRaNzCM1jZQQWbfvIt1wVEU+Y7WJBgwkQwls3Iv+fV+/K20mvig5ldJxYRueqKKqExSA=="}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:07.831795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:07.843137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-c24242e5-0ec9-41a5-befd-6b5513747040 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for project 75293bf0100f435bb567b690edc85533. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:05:07.847519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-c24242e5-0ec9-41a5-befd-6b5513747040 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for user 9a9256b6d7e24c54b061673cd17217cd and project 75293bf0100f435bb567b690edc85533. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:05:07.871889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c24242e5-0ec9-41a5-befd-6b5513747040 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.055635 Jul 24 01:05:07.872822 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 296/601] 10.222.0.18 () {62 vars in 1199 bytes} [Sun Jul 24 01:05:07 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 58 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 24 01:05:09.176922 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-719894545", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e240c1c7-87a5-4124-b96b-a74d28ba2a48"}], "security_groups": [{"name": "tempest-securitygroup--1410027683"}], "key_name": "tempest-keypair-274235092", "user_data": "}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:09.288162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] validate_networks() for [('e240c1c7-87a5-4124-b96b-a74d28ba2a48', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:05:09.430022 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:09.431274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:09.438033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:09.438399 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:09.439222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:09.440175 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:09.525458 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f267697f-4a0c-4f1d-9209-db1247e8a68d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/action" status: 202 len: 0 microversion: 2.1 time: 0.100115 Jul 24 01:05:09.526764 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 297/602] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:05:09 2022] POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/action => generated 0 bytes in 103 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:05:09.537428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:09.538291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:09.547094 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:09.547592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:09.548476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:09.549758 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:09.878476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:09.888001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-329f31fa-0220-4ccf-ac56-5f9c429ca264 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1753 microversion: 2.1 time: 0.356255 Jul 24 01:05:09.888596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 298/603] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:09 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1753 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:10.271536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:05:10.272116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:05:10.272680 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:05:10.273092 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:05:10.273509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:05:10.282385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:10.282873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:10.283820 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:10.284257 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:10.301633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for project 75293bf0100f435bb567b690edc85533. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:05:10.305632 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Getting quotas for user 9a9256b6d7e24c54b061673cd17217cd and project 75293bf0100f435bb567b690edc85533. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:05:10.317456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:05:10.320038 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:05:10.366516 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6007d49a-c7f4-44e4-b93a-24901044c4c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.194248 Jul 24 01:05:10.367195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 306/604] 10.222.0.18 () {62 vars in 1187 bytes} [Sun Jul 24 01:05:09 2022] POST /compute/v2.1/servers => generated 402 bytes in 1196 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 24 01:05:10.378266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-22291305-03a6-4b92-a773-6dc6512d5b38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:10.379143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-22291305-03a6-4b92-a773-6dc6512d5b38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:10.450957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-22291305-03a6-4b92-a773-6dc6512d5b38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:10.459075 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-22291305-03a6-4b92-a773-6dc6512d5b38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:10.460690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-22291305-03a6-4b92-a773-6dc6512d5b38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:10.461835 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-22291305-03a6-4b92-a773-6dc6512d5b38 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1280 microversion: 2.20 time: 0.088511 Jul 24 01:05:10.462391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 299/605] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:10 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1280 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:10.907309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:10.907923 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:10.916172 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:10.916647 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:10.917450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:10.917886 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:11.062360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:11.071856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-76af3616-11e4-404d-9295-858997267eda tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.169404 Jul 24 01:05:11.072419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 307/606] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:10 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:11.480833 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:11.481389 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:11.489202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:11.489657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:11.490419 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:11.490893 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:11.553487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:11.561457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:11.562988 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:11.564077 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-65cbb896-e291-4d69-a8d4-ada8a322d2c4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1332 microversion: 2.20 time: 0.087322 Jul 24 01:05:11.564770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 300/607] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:11 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1332 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:12.089415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:12.089941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:12.097339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:12.097841 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:12.098561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:12.099025 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:12.337822 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:12.346003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ef10d143-388e-4d65-bbf1-b8253e871588 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.260434 Jul 24 01:05:12.346496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 308/608] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:12 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:12.582590 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:12.583324 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:12.590361 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:12.591008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:12.591863 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:12.592370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:12.848055 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:12.856119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:12.857587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:12.858718 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ed967f4f-9df9-4f66-bfa2-82de97d8bf0d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.280657 Jul 24 01:05:12.859306 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 301/609] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:12 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:13.256766 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-1fb64fc1-b84a-4599-9b88-aff4231d3620 req-3816ce47-e376-4ee2-b656-13cd2e190db6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:13.278678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1fb64fc1-b84a-4599-9b88-aff4231d3620 req-3816ce47-e376-4ee2-b656-13cd2e190db6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.279158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-1fb64fc1-b84a-4599-9b88-aff4231d3620 req-3816ce47-e376-4ee2-b656-13cd2e190db6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.292186 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-1fb64fc1-b84a-4599-9b88-aff4231d3620 req-3816ce47-e376-4ee2-b656-13cd2e190db6 service nova] Creating event network-changed:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:05:13.297131 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-1fb64fc1-b84a-4599-9b88-aff4231d3620 req-3816ce47-e376-4ee2-b656-13cd2e190db6 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053150 Jul 24 01:05:13.297739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 309/610] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:13.363478 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:13.364184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:13.370684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.371184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.371981 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.372423 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.638701 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:13.648192 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:13.648576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3080cb1b-6b68-40b2-b64d-f7ff53a7d135 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.289414 Jul 24 01:05:13.649146 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 302/611] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:13 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:13.649292 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:13.655664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.656504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.657469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.657903 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.694471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1f8be96f-4f90-473e-bd32-4079da324317 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050573 Jul 24 01:05:13.695456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 310/612] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:05:13 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 24 01:05:13.753640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:13.754217 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:13.760231 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.760746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.761596 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.762093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.788619 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:05:13.863395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-06121135-b425-404d-b838-412738f4a9a8 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 204 len: 0 microversion: 2.1 time: 0.114443 Jul 24 01:05:13.863969 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 303/613] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:05:13 2022] DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:05:13.872341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:13.872967 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:13.878614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:13.879348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:13.879999 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.880525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.881500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.881940 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.886222 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.886678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:13.887488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:13.887956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:14.078751 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:14.085633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:14.086678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-affd96af-c8b5-4555-8e20-22ef7e37d6cb tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1674 microversion: 2.1 time: 0.218339 Jul 24 01:05:14.087329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 311/614] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:13 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1674 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:14.093615 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:14.095179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:14.096948 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a05c9313-cfeb-457d-a03b-734bc63e0ed3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.223457 Jul 24 01:05:14.097494 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 304/615] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:13 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:05:14.666745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:14.667428 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:14.674484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:14.674982 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:14.675747 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:14.676172 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:14.691162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:14.819209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:14.827007 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3917c3b4-79ae-45a6-a5b2-3f411e9886b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.164113 Jul 24 01:05:14.827567 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 312/616] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:14 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:15.104895 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:15.105432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:15.113006 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:15.113554 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:15.114418 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:15.114966 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:15.115099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:15.115602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:15.121648 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:15.122093 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:15.122884 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:15.123320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:15.309290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:15.317066 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:15.317825 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9b6fe1f9-cd53-4561-97db-bcddb12f6521 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1674 microversion: 2.1 time: 0.216895 Jul 24 01:05:15.318408 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 305/617] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:15 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1674 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:15.326278 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:15.327820 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:15.328948 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7c3f988c-5bc3-43b4-81ae-74372f335254 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.218271 Jul 24 01:05:15.329505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 313/618] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:15 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:05:15.844299 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:15.844910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:15.851125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:15.851577 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:15.852345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:15.852781 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:15.993857 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:16.001800 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1d66f79c-91c5-482b-b4c2-bdaaea372e01 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.161314 Jul 24 01:05:16.002372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 306/619] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:15 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:16.335552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:16.336209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:16.342837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:16.343292 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:16.344058 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:16.344491 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:16.347454 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:16.348136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:16.354322 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:16.354785 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:16.355561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:16.355992 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:16.542228 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:16.546042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:16.550170 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-52a75fb7-b0ce-4cbd-b367-78dc448ddd2f tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1674 microversion: 2.1 time: 0.218342 Jul 24 01:05:16.550879 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 314/620] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:16 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1674 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:16.554163 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:16.555707 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:16.556796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fb809565-66c5-4cfa-9192-7a294c5ae1dc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.213989 Jul 24 01:05:16.557352 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 307/621] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:16 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:05:17.021009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:17.022014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:17.028708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:17.029302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:17.030195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:17.030736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:17.183800 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:17.193144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4876f92b-107d-4f47-986b-d7b3174610eb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.177275 Jul 24 01:05:17.194198 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 315/622] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:17 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:17.567934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:17.569767 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:17.575227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:17.576467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:17.588435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:17.589138 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:17.590145 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:17.590823 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:17.596192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:17.600987 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:17.601929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:17.602397 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:18.126037 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:18.135457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2b224db7-bfe9-4b0a-bc23-d0bcc5c71c1d tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 200 len: 1674 microversion: 2.1 time: 0.571944 Jul 24 01:05:18.145993 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 308/623] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:17 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 1674 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:18.156794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:18.166589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:18.168374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:18.169342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ccaef2ec-f56a-4562-a027-e39c5698fdcc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.597935 Jul 24 01:05:18.169937 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 316/624] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:17 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:05:18.209455 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:18.210064 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:18.216098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:18.216549 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:18.217311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:18.217749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:18.368866 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:18.377935 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8de93b33-9285-4e2e-85d8-7b548dade440 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.172267 Jul 24 01:05:18.378377 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 309/625] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:18 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:18.543403 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-00a4847b-ec3e-4c58-b078-654996d65bde req-47af5c2c-aea6-4fe7-8010-1476d0251bfc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64ba3b57-16f8-4b6f-b0a6-d0141afc0faa", "name": "network-vif-deleted", "tag": "710ccda1-d8a7-4b65-9124-7d15ed6b5d75"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:18.555906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-00a4847b-ec3e-4c58-b078-654996d65bde req-47af5c2c-aea6-4fe7-8010-1476d0251bfc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:18.557026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-00a4847b-ec3e-4c58-b078-654996d65bde req-47af5c2c-aea6-4fe7-8010-1476d0251bfc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:18.572920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-00a4847b-ec3e-4c58-b078-654996d65bde req-47af5c2c-aea6-4fe7-8010-1476d0251bfc service nova] Creating event network-vif-deleted:710ccda1-d8a7-4b65-9124-7d15ed6b5d75 for instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa on hv-cinder-maste Jul 24 01:05:18.579928 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-00a4847b-ec3e-4c58-b078-654996d65bde req-47af5c2c-aea6-4fe7-8010-1476d0251bfc service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039668 Jul 24 01:05:18.582457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 317/626] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:19.152870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:19.153482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:19.167017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.167479 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.168265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.168874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.184779 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] HTTP exception thrown: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Request method failure captured: Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa HTTP/1.1 Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm8nnZce8yp0GRDlYqUl0iNdKlyJb5uKKi3sf1HvzqIkN_s6psHRHdLLUWihPvdD8IoiN0VH_RTioVAkBC3TTe8ckleScL7HRvbfiaqAxJGBSZqKAG7x_HZIcGDyreen0LNHvSjWCRPLgXWzIa1ypoyelWd3YaaB39JfrVeWNJQJuw Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:05:19.186896 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_644464d3831e468f8ab9957bb33817b0", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: a59f5944a0db4710a43a6447019864d2 Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:05:19.188152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '64ba3b57-16f8-4b6f-b0a6-d0141afc0faa'} Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:05:19.188881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:05:19.190557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:19.191634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:19.191634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:19.194001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Returning 404 to user: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:05:19.195007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d2145085-30a0-4e08-a3cc-864894747e28 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 404 len: 111 microversion: 2.1 time: 0.045853 Jul 24 01:05:19.199208 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 310/627] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:19 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:05:19.205621 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f4d13592-0c2c-4ff1-bf72-5547aefba18b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '.version_select at 0x7f5e3038a940>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:19.222140 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.224733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.227232 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f4d13592-0c2c-4ff1-bf72-5547aefba18b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1776576010" status: 202 len: 0 microversion: 2.1 time: 0.026469 Jul 24 01:05:19.227999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 311/628] 10.222.0.18 () {58 vars in 1220 bytes} [Sun Jul 24 01:05:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1776576010 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:05:19.230449 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.230983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.257413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-dd364fc6-18cf-4fdd-9eab-36c76821f01b req-5c8a7ecb-6c8d-4e36-ae0c-fecd3bc905fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "name": "network-vif-plugged", "status": "completed", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:19.267204 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-dd364fc6-18cf-4fdd-9eab-36c76821f01b req-5c8a7ecb-6c8d-4e36-ae0c-fecd3bc905fd service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.268910 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-dd364fc6-18cf-4fdd-9eab-36c76821f01b req-5c8a7ecb-6c8d-4e36-ae0c-fecd3bc905fd service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.293728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-dd364fc6-18cf-4fdd-9eab-36c76821f01b req-5c8a7ecb-6c8d-4e36-ae0c-fecd3bc905fd service nova] Creating event network-vif-plugged:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:05:19.300613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-dd364fc6-18cf-4fdd-9eab-36c76821f01b req-5c8a7ecb-6c8d-4e36-ae0c-fecd3bc905fd service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046719 Jul 24 01:05:19.301859 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 312/629] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:19.401208 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:19.401907 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:19.409244 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.409830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.410630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:19.421759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:19.425512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:19.430740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:19.433319 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:19.435177 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:19.436570 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c0ba97f1-da42-4d19-b1cd-a85cfc943b77 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.252094 Jul 24 01:05:19.443398 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 318/630] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:19 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:19.607140 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:19.616317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-70ec933f-cba4-489b-a6c8-229dc9c889b8 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.224904 Jul 24 01:05:19.617074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 313/631] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:19 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:19.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-91683b7e-b6cd-43f4-a78c-5a6eb7c217e2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: .version_select at 0x7f5e305983a0>, body: {"keypair": {"name": "tempest-keypair-766938946", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJYYhTstPop/Au2bws0TLzE1TXkxF+AQZj5ICPf1YHYykvUtmfH2c4pCtaE2d9b1WNjvwD1xhaGxOHxmsJbPWIWjiKzDn4BSIj0bHwPH8mWMErJT6poFpBgLR/oyJZ+c0Q=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:20.004931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-91683b7e-b6cd-43f4-a78c-5a6eb7c217e2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for project 644464d3831e468f8ab9957bb33817b0. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:05:20.008695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-91683b7e-b6cd-43f4-a78c-5a6eb7c217e2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for user a59f5944a0db4710a43a6447019864d2 and project 644464d3831e468f8ab9957bb33817b0. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:05:20.030834 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-91683b7e-b6cd-43f4-a78c-5a6eb7c217e2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.044438 Jul 24 01:05:20.031523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 319/632] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:05:19 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:20.460126 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:20.460740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:20.463059 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:20.468526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:20.468987 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:20.469762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:20.470726 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:20.630790 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:20.636067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:20.636850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:20.643502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:20.643963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:20.644741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:20.645182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:20.683671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:20.692049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:20.693838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:20.695049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9ceae4b1-a5b0-4fd8-b9b7-e5734c28dd7f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1406 microversion: 2.20 time: 0.238735 Jul 24 01:05:20.695754 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 314/633] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:20 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1406 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:05:20.893701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:20.902294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-403b75b3-8463-4987-941e-3762a31ae047 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.272843 Jul 24 01:05:20.902918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 320/634] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:20 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:21.385117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-731286844", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d2871582-ae5b-4292-87a4-e519f553496c"}], "security_groups": [{"name": "tempest-securitygroup--1778934033"}], "key_name": "tempest-keypair-766938946", "user_data": "}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:21.489882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] validate_networks() for [('d2871582-ae5b-4292-87a4-e519f553496c', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:05:21.713230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:21.713897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:21.720963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:21.721479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:21.722247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:21.722675 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:21.925026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:21.932924 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:21.934415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:21.935585 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c6b3b160-b5dd-4ff9-891f-49a810bbd490 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1598 microversion: 2.20 time: 0.226301 Jul 24 01:05:21.936134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 321/635] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:05:21 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1598 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:05:21.940592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:05:21.941018 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:05:21.941685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:05:21.941823 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:21.942105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:05:21.942330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:21.942516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:05:21.948439 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:21.948884 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:21.949644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:21.950071 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:21.952120 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:21.952669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:21.953541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:21.954050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:21.971737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for project 644464d3831e468f8ab9957bb33817b0. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:05:21.975831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Getting quotas for user a59f5944a0db4710a43a6447019864d2 and project 644464d3831e468f8ab9957bb33817b0. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:05:21.987383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:05:21.989681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:05:22.037197 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-89acd3e3-4a10-4187-a2af-90fae8f564b4 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.656163 Jul 24 01:05:22.042685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 315/636] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:05:21 2022] POST /compute/v2.1/servers => generated 402 bytes in 657 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:05:22.042685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:22.043162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:22.049204 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:22.049568 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:22.050345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:22.050802 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:22.141260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:22.150231 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9a7d552e-d9c6-44de-a559-27c81e9846b4 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.213106 Jul 24 01:05:22.160552 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 322/637] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:21 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:22.161637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c8101ed3-6ab0-42e4-9666-97f2165144cb tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.115484 Jul 24 01:05:22.161983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a2ee2ed2-cc39-4ab0-9c04-87990004a492 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:22.161983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a2ee2ed2-cc39-4ab0-9c04-87990004a492 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:22.162339 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 316/638] 10.222.0.18 () {60 vars in 1315 bytes} [Sun Jul 24 01:05:22 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-interface => generated 284 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:05:22.266442 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a2ee2ed2-cc39-4ab0-9c04-87990004a492 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:22.274564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a2ee2ed2-cc39-4ab0-9c04-87990004a492 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1236 microversion: 2.1 time: 0.122458 Jul 24 01:05:22.275262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 323/639] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:22 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1236 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:23.125687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-95b1e22c-a3d4-405b-98d1-13b4e6a360c9 req-6c6e3c03-9b1a-4f08-9bcc-645e96613f3a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:23.134683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-95b1e22c-a3d4-405b-98d1-13b4e6a360c9 req-6c6e3c03-9b1a-4f08-9bcc-645e96613f3a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:23.135173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-95b1e22c-a3d4-405b-98d1-13b4e6a360c9 req-6c6e3c03-9b1a-4f08-9bcc-645e96613f3a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:23.146874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-95b1e22c-a3d4-405b-98d1-13b4e6a360c9 req-6c6e3c03-9b1a-4f08-9bcc-645e96613f3a service nova] Creating event network-changed:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:05:23.151970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-95b1e22c-a3d4-405b-98d1-13b4e6a360c9 req-6c6e3c03-9b1a-4f08-9bcc-645e96613f3a service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030342 Jul 24 01:05:23.152740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 317/640] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:23.168160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:23.168954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:23.177655 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:23.178105 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:23.178944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:23.179374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:23.291942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:23.292642 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:23.306854 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:23.307621 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:23.308803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:23.309765 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:23.435906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:23.445226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b618d62b-ceda-4149-a896-b3ee8d8d819e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1294 microversion: 2.1 time: 0.157405 Jul 24 01:05:23.446363 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 318/641] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:23 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1294 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:23.499925 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:23.508061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-99c4f7a3-92f5-4b80-b76f-af7864f79efe tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.343756 Jul 24 01:05:23.508714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 324/642] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:23 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:24.461608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:24.462244 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:24.469126 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:24.469580 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:24.470566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:24.471021 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:24.525473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:24.526100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:24.542017 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:24.542471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:24.543287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:24.543730 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:24.701121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:24.710315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4f37143f-2eec-4b01-a7a4-d18e04802235 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1362 microversion: 2.1 time: 0.252463 Jul 24 01:05:24.719671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 319/643] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:24 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1362 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:24.745035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:24.755000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ac9d0046-56f2-41ae-8390-a8a53198878e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.233278 Jul 24 01:05:24.757146 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 325/644] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:24 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:24.783375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-3e7f2c58-27fb-405e-8681-ea1af4f4f7af req-27d33339-167d-4174-b5ef-08a63a38400d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c4a54e8-8220-4b56-bbb7-51402654c743", "tag": "ffc72e66-3327-493d-a572-31822359f754"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:24.793759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-3e7f2c58-27fb-405e-8681-ea1af4f4f7af req-27d33339-167d-4174-b5ef-08a63a38400d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:24.794447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-3e7f2c58-27fb-405e-8681-ea1af4f4f7af req-27d33339-167d-4174-b5ef-08a63a38400d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:24.806302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-3e7f2c58-27fb-405e-8681-ea1af4f4f7af req-27d33339-167d-4174-b5ef-08a63a38400d service nova] Creating event network-changed:ffc72e66-3327-493d-a572-31822359f754 for instance 1c4a54e8-8220-4b56-bbb7-51402654c743 on hv-cinder-maste Jul 24 01:05:24.812340 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-3e7f2c58-27fb-405e-8681-ea1af4f4f7af req-27d33339-167d-4174-b5ef-08a63a38400d service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033635 Jul 24 01:05:24.812951 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 320/645] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:25.727522 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:25.728095 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:25.734141 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:25.734592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:25.735386 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:25.736546 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:25.773860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:25.774562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:25.781945 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:25.782412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:25.783730 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:25.784169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:25.916313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:25.924665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-299dcffc-b9c0-487d-b927-7a0b1c1002aa tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1362 microversion: 2.1 time: 0.201178 Jul 24 01:05:25.925244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 326/646] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:25 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1362 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:25.955452 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:25.963280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3582e197-35bf-415f-b948-f0f9fb3f481e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.193612 Jul 24 01:05:25.963841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 321/647] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:25 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:26.942132 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:26.942737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:26.949557 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:26.949998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:26.950742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:26.951194 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:26.980890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:26.981501 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:26.989032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:26.989483 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:26.990303 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:26.990688 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:27.139173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:27.149665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b674265d-6a94-43fd-8454-8427c97857b3 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1362 microversion: 2.1 time: 0.211993 Jul 24 01:05:27.150257 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 327/648] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:26 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1362 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:27.187476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:27.196086 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5497caab-59cc-4bd3-b4a2-75eb5c95aa35 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.218966 Jul 24 01:05:27.196648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 322/649] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:26 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:28.167011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:28.167667 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:28.174114 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:28.174566 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:28.175365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:28.175802 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:28.213127 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:28.213731 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:28.220726 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:28.221179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:28.221964 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:28.222425 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:28.597860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:28.605338 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e1db6886-e179-4e97-9fa2-6323898cf12e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1362 microversion: 2.1 time: 0.442219 Jul 24 01:05:28.606161 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 328/650] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:28 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1362 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:28.645488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:28.653490 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-40020f50-e29a-41cf-ad61-21f0015a1d8a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.444139 Jul 24 01:05:28.654052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 323/651] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:28 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:29.183916 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-5b289651-c174-4a31-b10c-07893662ee8d req-ee96db1f-515b-4adf-93b8-5e23c457aa37 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c4a54e8-8220-4b56-bbb7-51402654c743", "name": "network-vif-plugged", "status": "completed", "tag": "ffc72e66-3327-493d-a572-31822359f754"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:29.192581 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-5b289651-c174-4a31-b10c-07893662ee8d req-ee96db1f-515b-4adf-93b8-5e23c457aa37 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:29.193032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-5b289651-c174-4a31-b10c-07893662ee8d req-ee96db1f-515b-4adf-93b8-5e23c457aa37 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:29.204901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-5b289651-c174-4a31-b10c-07893662ee8d req-ee96db1f-515b-4adf-93b8-5e23c457aa37 service nova] Creating event network-vif-plugged:ffc72e66-3327-493d-a572-31822359f754 for instance 1c4a54e8-8220-4b56-bbb7-51402654c743 on hv-cinder-maste Jul 24 01:05:29.209959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-5b289651-c174-4a31-b10c-07893662ee8d req-ee96db1f-515b-4adf-93b8-5e23c457aa37 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029716 Jul 24 01:05:29.210876 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 329/652] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:29.623435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:29.624048 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:29.630897 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:29.631346 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:29.632099 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:29.632532 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:29.671635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:29.672472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:29.679006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:29.679458 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:29.680230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:29.680662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:29.809419 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:29.817724 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a9e6bab0-6dd5-409d-b7cb-1e4deefb88a2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1362 microversion: 2.1 time: 0.198063 Jul 24 01:05:29.818296 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 324/653] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:29 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1362 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:29.854417 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:29.863034 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-87e66b00-b11b-4ad7-96a0-d39e58bdeefc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.195618 Jul 24 01:05:29.863604 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 330/654] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:29 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:30.835134 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:30.835746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:30.842216 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:30.842667 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:30.843473 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:30.843913 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:30.881939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:30.882471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:30.889127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:30.889641 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:30.890471 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:30.890996 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:31.023284 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:31.030665 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2d2ac6c4-2927-453b-a645-047c989fbdee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1362 microversion: 2.1 time: 0.199438 Jul 24 01:05:31.036355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 325/655] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:30 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1362 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:31.068301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:31.076543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-83fef3e4-9db2-4e85-99cc-c5c130103c6a tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.199879 Jul 24 01:05:31.077112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 331/656] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:30 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:32.046657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:32.047295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:32.050516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:32.051213 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:32.056914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:32.057369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:32.058144 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:32.058583 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:32.094640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:32.096036 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:32.102669 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:32.103149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:32.103918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:32.104362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:32.119674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=100 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:32.235319 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:32.242967 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9362e8a3-cd29-4fd5-bc5c-4b957277f9e5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1547 microversion: 2.1 time: 0.200121 Jul 24 01:05:32.255570 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 326/657] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:32 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1547 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:32.260657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:32.261150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:32.267784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:32.268242 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:32.269012 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:32.269453 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:32.271534 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=12 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:32.291161 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:32.299987 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-34b2d8fb-53f6-46c5-9b74-ebf31e53059e tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.208970 Jul 24 01:05:32.300858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 332/658] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:32 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:32.351960 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-232cee43-3ac5-44d2-b6f3-bf042f55c2ba tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.095131 Jul 24 01:05:32.352608 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 327/659] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:05:32 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:05:33.270170 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-4524d09e-5b2a-4935-9c4e-3e6b1f59b814 req-f449525b-1416-43b0-9873-27d5c19c59d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c4a54e8-8220-4b56-bbb7-51402654c743", "tag": "ffc72e66-3327-493d-a572-31822359f754"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:33.278601 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-4524d09e-5b2a-4935-9c4e-3e6b1f59b814 req-f449525b-1416-43b0-9873-27d5c19c59d7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:33.279073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-4524d09e-5b2a-4935-9c4e-3e6b1f59b814 req-f449525b-1416-43b0-9873-27d5c19c59d7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:33.290034 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-4524d09e-5b2a-4935-9c4e-3e6b1f59b814 req-f449525b-1416-43b0-9873-27d5c19c59d7 service nova] Creating event network-changed:ffc72e66-3327-493d-a572-31822359f754 for instance 1c4a54e8-8220-4b56-bbb7-51402654c743 on hv-cinder-maste Jul 24 01:05:33.294748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-4524d09e-5b2a-4935-9c4e-3e6b1f59b814 req-f449525b-1416-43b0-9873-27d5c19c59d7 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028978 Jul 24 01:05:33.295722 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 333/660] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:33.318265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:33.319175 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:33.325357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:33.325812 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:33.326589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:33.327128 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:33.491137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:33.503333 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-db46352c-960f-4401-8e62-1ae6dae4ffed tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1758 microversion: 2.1 time: 0.189944 Jul 24 01:05:33.513029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 328/661] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:33 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1758 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:34.522261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:34.522917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:34.529189 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:34.529638 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:34.530405 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:34.530874 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:34.832269 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:34.840798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c3b5670f-33f5-48d9-b60f-a07ba82734d9 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1761 microversion: 2.1 time: 0.323319 Jul 24 01:05:34.845392 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 334/662] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:34 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1761 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:38.083006 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:38.083533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:38.085855 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=4,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:38.091464 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:38.091918 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:38.092684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:38.093123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:38.274678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lazy-loading 'services' on Instance uuid c6a71fa6-32c2-446f-b9e5-80d4e9ea800e {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:05:38.348649 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-15919526-d12e-4013-8ef8-9c1be65f3542 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments/d1a3d8e7-19a4-40bf-a311-17e785b4bf50" status: 202 len: 0 microversion: 2.1 time: 0.268968 Jul 24 01:05:38.349759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 329/663] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:05:38 2022] DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments/d1a3d8e7-19a4-40bf-a311-17e785b4bf50 => generated 0 bytes in 271 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:05:38.357900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:38.358858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:38.361896 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=4 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:38.367305 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:38.367728 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:38.368502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:38.369073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:38.400027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-03ac7a26-ae0c-458c-b127-ccc7265429c4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045745 Jul 24 01:05:38.400447 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 335/664] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:05:38 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:39.415462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:39.415462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:39.424049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:39.424391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:39.425246 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:39.425695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:39.457506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5bfe3d21-1578-4efa-87b5-2d91cd486277 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049146 Jul 24 01:05:39.458083 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 330/665] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:05:39 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:40.469109 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:40.470063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:40.476732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:40.477181 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:40.477946 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:40.478382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:40.507428 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2c89808e-a088-44cf-9206-93d2bfd8564a tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042235 Jul 24 01:05:40.507910 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 336/666] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:05:40 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:05:40.586149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:40.586683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:40.593108 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:40.593558 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:40.594340 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:40.594794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:40.620805 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:05:40.623547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:05:40.694516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-222835f8-07e3-4f28-9f02-37d5a64b9b55 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 204 len: 0 microversion: 2.1 time: 0.111910 Jul 24 01:05:40.695142 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 331/667] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:05:40 2022] DELETE /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:05:40.703317 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:40.704104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:40.710271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:40.710702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:40.711496 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:40.711930 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:40.843429 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:40.851684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6e6ff541-84ed-478f-b319-90db5754bb5b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1686 microversion: 2.1 time: 0.152237 Jul 24 01:05:40.852247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 337/668] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:40 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1686 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:41.869968 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:41.870690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:41.877711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:41.878153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:41.878942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:41.879377 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:42.005294 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:42.012771 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a6770554-a24f-4abc-ace1-f0b9270810c2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1686 microversion: 2.1 time: 0.147042 Jul 24 01:05:42.013331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 332/669] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:41 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1686 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:43.029838 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:43.030458 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:43.037271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:43.037732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:43.038491 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:43.038963 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:43.165081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:43.172511 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ab4bcdbf-0c28-4e42-83e0-493a90740e62 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1686 microversion: 2.1 time: 0.146598 Jul 24 01:05:43.173075 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 338/670] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:43 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1686 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:44.189655 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:44.190179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:44.197169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:44.197624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:44.198392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:44.198857 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:44.334344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:44.341929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7cfe3635-0c40-49e2-83fa-52e42189bd63 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1686 microversion: 2.1 time: 0.156437 Jul 24 01:05:44.342486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 333/671] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:44 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1686 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:45.359197 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:45.359809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:45.368145 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:45.368607 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:45.369625 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:45.370067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:45.510384 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:45.518929 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f891c5ec-f6a7-461b-aa43-2fc8edd141f4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1686 microversion: 2.1 time: 0.163627 Jul 24 01:05:45.519521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 339/672] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:45 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1686 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:45.927278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-4f74427f-8fcd-473b-bf9e-3ace6b6b61ce req-f13dbe75-bdec-4630-93d7-7a34a8fe38ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c6a71fa6-32c2-446f-b9e5-80d4e9ea800e", "name": "network-vif-deleted", "tag": "c3604b8e-6ace-40d2-8c7a-9f222b26d78c"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:45.937351 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-4f74427f-8fcd-473b-bf9e-3ace6b6b61ce req-f13dbe75-bdec-4630-93d7-7a34a8fe38ea service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:45.938092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-4f74427f-8fcd-473b-bf9e-3ace6b6b61ce req-f13dbe75-bdec-4630-93d7-7a34a8fe38ea service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:45.951331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-4f74427f-8fcd-473b-bf9e-3ace6b6b61ce req-f13dbe75-bdec-4630-93d7-7a34a8fe38ea service nova] Creating event network-vif-deleted:c3604b8e-6ace-40d2-8c7a-9f222b26d78c for instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e on hv-cinder-maste Jul 24 01:05:45.965429 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-4f74427f-8fcd-473b-bf9e-3ace6b6b61ce req-f13dbe75-bdec-4630-93d7-7a34a8fe38ea service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041749 Jul 24 01:05:45.966595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 334/673] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:46.537598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:46.538243 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:46.552935 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:46.553298 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:46.554076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:46.554512 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:46.587097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:46.587764 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:46.588372 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lazy-loading 'fault' on Instance uuid c6a71fa6-32c2-446f-b9e5-80d4e9ea800e {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:05:46.642637 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:46.650347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0e85f0fa-8af0-47a6-bf3b-0072d190e4d5 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 200 len: 1335 microversion: 2.1 time: 0.117931 Jul 24 01:05:46.650966 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 340/674] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:46 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 1335 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:47.669259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:47.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: c6a71fa6-32c2-446f-b9e5-80d4e9ea800e] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:47.677034 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:47.677484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:47.678258 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:47.678723 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:47.691946 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] HTTP exception thrown: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e could not be found. Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Request method failure captured: Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e HTTP/1.1 Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnFiejBFZa8mJ5D502lXi0vI2AJZSXzLQ6yvLVNBpg5wuvRciV8GsNNEfmF5OiwQV_hyTlbh1vvTV_RBBoRfFvSvMn5HPevJqiFZ0JTvv9BtvBk2xvVAY9_4RTM-tU7gCIt6rWRh-Ok03kE5HtQWgevEOi1XOD-L9IgzsF_0g4DZ8U Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:05:47.693797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_f7d219e82efe44f9b6f860a5fabc7733", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 283a1bfe0e3b45aaa0a33e5fe6b1d85a Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e could not be found. Jul 24 01:05:47.695422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'c6a71fa6-32c2-446f-b9e5-80d4e9ea800e'} Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e could not be found. Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:05:47.696484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e could not be found. Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e could not be found. Jul 24 01:05:47.698031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:05:47.699566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Returning 404 to user: Instance c6a71fa6-32c2-446f-b9e5-80d4e9ea800e could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:05:47.699566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bea1eb4b-001d-4910-b923-db9111036635 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e" status: 404 len: 111 microversion: 2.1 time: 0.030722 Jul 24 01:05:47.699566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 335/675] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:47 2022] GET /compute/v2.1/servers/c6a71fa6-32c2-446f-b9e5-80d4e9ea800e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:05:47.703793 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a36697e5-8bb7-4ab2-a050-6fb96f53def0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '.version_select at 0x7f5e34d91940>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:47.720965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a36697e5-8bb7-4ab2-a050-6fb96f53def0 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1290565022" status: 202 len: 0 microversion: 2.1 time: 0.020925 Jul 24 01:05:47.721506 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 341/676] 10.222.0.18 () {58 vars in 1220 bytes} [Sun Jul 24 01:05:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1290565022 => generated 0 bytes in 22 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:05:50.553578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:50.554070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:50.560523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:50.560999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:50.561761 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:50.562195 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:50.587086 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:05:50.662019 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-27364685-136f-4588-b5dd-483dbb7344b4 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "DELETE /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 204 len: 0 microversion: 2.1 time: 0.111746 Jul 24 01:05:50.663297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 336/677] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:05:50 2022] DELETE /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:05:50.671506 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:50.672202 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:50.679421 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:50.679871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:50.680640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:50.681077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:50.808819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:50.816560 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-91990b28-eb2f-40e1-8a44-ac15e4778621 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1507 microversion: 2.1 time: 0.149471 Jul 24 01:05:50.817130 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 342/678] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:50 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1507 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:51.030034 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:51.030531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:51.037462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:51.037904 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:51.038660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:51.039119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:51.064956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-849332ab-93eb-4bb2-8da8-1d53d74dcdd0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/ips" status: 200 len: 149 microversion: 2.1 time: 0.039198 Jul 24 01:05:51.065504 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 337/679] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:05:51 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/ips => generated 149 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:51.833537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:51.834136 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:51.840516 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:51.840960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:51.841709 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:51.842166 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:51.968482 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:51.975991 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-83c86f3a-c92d-450a-8eb5-65ba5b7a9093 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1507 microversion: 2.1 time: 0.146434 Jul 24 01:05:51.976561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 343/680] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:51 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1507 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:52.376653 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e043d420-cfbc-4fb9-9026-40328d94c4bb"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:52.377961 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:52.384499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:52.384950 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:52.385737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:52.386171 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:52.993504 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:52.994018 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:53.000488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:53.000939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:53.001699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:53.002160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:53.129370 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:53.136778 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0a0bb63f-4e53-45e4-aa28-fe606ff9e62b tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1507 microversion: 2.1 time: 0.147239 Jul 24 01:05:53.137339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 344/681] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:52 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1507 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:05:53.317256 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d77794ad-4a0d-4c6c-b079-e90871061112 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.944347 Jul 24 01:05:53.317843 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 338/682] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:05:52 2022] POST /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments => generated 194 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:54.044661 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-b316f384-589a-438c-bbe9-5b5123066602 req-b2c46843-64af-471c-a639-63e6849afcae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4389fd86-dd98-40ca-8e28-a9153502931c", "name": "network-vif-deleted", "tag": "dd19007e-705d-4543-9981-8c458087a8e3"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:54.058076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-b316f384-589a-438c-bbe9-5b5123066602 req-b2c46843-64af-471c-a639-63e6849afcae service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:54.058758 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-b316f384-589a-438c-bbe9-5b5123066602 req-b2c46843-64af-471c-a639-63e6849afcae service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:54.074846 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-b316f384-589a-438c-bbe9-5b5123066602 req-b2c46843-64af-471c-a639-63e6849afcae service nova] Creating event network-vif-deleted:dd19007e-705d-4543-9981-8c458087a8e3 for instance 4389fd86-dd98-40ca-8e28-a9153502931c on hv-cinder-maste Jul 24 01:05:54.082599 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-b316f384-589a-438c-bbe9-5b5123066602 req-b2c46843-64af-471c-a639-63e6849afcae service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041244 Jul 24 01:05:54.083535 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 345/683] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:05:54.155827 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:54.157027 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:54.163881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:54.164333 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:54.165100 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:54.165535 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:54.228640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:54.236119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cd2f49ba-1c39-4241-9e13-58f0662f5926 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 200 len: 1292 microversion: 2.1 time: 0.085062 Jul 24 01:05:54.236765 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 339/684] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:54 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 1292 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:55.308329 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:55.309028 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 4389fd86-dd98-40ca-8e28-a9153502931c] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:55.315402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:55.315858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:55.316624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:55.317068 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:55.330510 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] HTTP exception thrown: Instance 4389fd86-dd98-40ca-8e28-a9153502931c could not be found. Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Request method failure captured: Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c HTTP/1.1 Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnFiejBFZa8mJ5D502lXi0vI2AJZSXzLQ6yvLVNBpg5wuvRciV8GsNNEfmF5OiwQV_hyTlbh1vvTV_RBBoRfFvSvMn5HPevJqiFZ0JTvv9BtvBk2xvVAY9_4RTM-tU7gCIt6rWRh-Ok03kE5HtQWgevEOi1XOD-L9IgzsF_0g4DZ8U Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:05:55.332173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_f7d219e82efe44f9b6f860a5fabc7733", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 283a1bfe0e3b45aaa0a33e5fe6b1d85a Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 4389fd86-dd98-40ca-8e28-a9153502931c could not be found. Jul 24 01:05:55.333711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '4389fd86-dd98-40ca-8e28-a9153502931c'} Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4389fd86-dd98-40ca-8e28-a9153502931c could not be found. Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:05:55.334578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4389fd86-dd98-40ca-8e28-a9153502931c could not be found. Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4389fd86-dd98-40ca-8e28-a9153502931c could not be found. Jul 24 01:05:55.336234 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:05:55.337394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Returning 404 to user: Instance 4389fd86-dd98-40ca-8e28-a9153502931c could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:05:55.337394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b42c9004-91af-4ad4-97e6-6cc6f556e74e tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c" status: 404 len: 111 microversion: 2.1 time: 0.084234 Jul 24 01:05:55.337394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 346/685] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:55 2022] GET /compute/v2.1/servers/4389fd86-dd98-40ca-8e28-a9153502931c => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:05:55.342206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:55.342702 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:55.349817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:55.350266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:55.351056 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:55.351492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:55.376404 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:05:55.449967 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-40a78e05-17c0-4fab-bf0a-612f84bc1096 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "DELETE /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 204 len: 0 microversion: 2.1 time: 0.111177 Jul 24 01:05:55.450833 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 340/686] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:05:55 2022] DELETE /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:05:55.459907 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:55.460521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:55.466537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:55.467023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:55.467794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:55.468231 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:55.599261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:55.606673 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9a43a936-909d-443d-9a56-74e1d77a4af2 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.151221 Jul 24 01:05:55.607260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 347/687] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:55 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:56.624534 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:56.625241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:56.631888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:56.632349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:56.633116 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:56.633558 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:56.759912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:56.767270 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6106c798-9951-44de-b723-0b8976def0dc tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.147005 Jul 24 01:05:56.769039 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 341/688] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:56 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:57.786003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:57.786525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:57.792976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:57.793419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:57.794171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:57.794600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:57.919067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:57.926410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-092b0f71-c643-4a49-9525-d46d03a01d58 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.144865 Jul 24 01:05:57.927000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 348/689] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:57 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:58.944410 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:05:58.945012 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:05:58.951674 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:58.952118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:58.952877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:58.953307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:59.083838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:05:59.091380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b969eff4-231b-4c47-a45b-48a96669cced tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1520 microversion: 2.1 time: 0.152047 Jul 24 01:05:59.091950 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 342/690] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:05:58 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1520 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:05:59.633247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-2e88a6e8-afa9-47a4-93ed-e6fc69480678 req-24479d8d-9374-4e14-ad63-f616939c58da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "97d6538b-2585-40fc-b68c-e9a088724a39", "name": "network-vif-deleted", "tag": "7813289d-8139-4d10-ab8e-926e1b2e6577"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:05:59.641665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2e88a6e8-afa9-47a4-93ed-e6fc69480678 req-24479d8d-9374-4e14-ad63-f616939c58da service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:05:59.642110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2e88a6e8-afa9-47a4-93ed-e6fc69480678 req-24479d8d-9374-4e14-ad63-f616939c58da service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:05:59.653826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-2e88a6e8-afa9-47a4-93ed-e6fc69480678 req-24479d8d-9374-4e14-ad63-f616939c58da service nova] Creating event network-vif-deleted:7813289d-8139-4d10-ab8e-926e1b2e6577 for instance 97d6538b-2585-40fc-b68c-e9a088724a39 on hv-cinder-maste Jul 24 01:05:59.660342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-2e88a6e8-afa9-47a4-93ed-e6fc69480678 req-24479d8d-9374-4e14-ad63-f616939c58da service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030145 Jul 24 01:05:59.661090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 349/691] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:05:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:00.108249 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:00.109541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:00.116657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:00.117108 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:00.117877 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:00.118306 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:00.189914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '283a1bfe0e3b45aaa0a33e5fe6b1d85a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7d219e82efe44f9b6f860a5fabc7733', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:00.192518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=5,nova_cell1:INSERT=4 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:00.199523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ff2ae2a9-7a87-4c9d-aca9-90aa80635fdb tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 200 len: 1306 microversion: 2.1 time: 0.095223 Jul 24 01:06:00.200087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 343/692] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:00 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 1306 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:01.217723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:01.218419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] [instance: 97d6538b-2585-40fc-b68c-e9a088724a39] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:01.225450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:01.225897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:01.226653 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:01.227111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:01.240976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] HTTP exception thrown: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 could not be found. Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Request method failure captured: Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 HTTP/1.1 Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnFiejBFZa8mJ5D502lXi0vI2AJZSXzLQ6yvLVNBpg5wuvRciV8GsNNEfmF5OiwQV_hyTlbh1vvTV_RBBoRfFvSvMn5HPevJqiFZ0JTvv9BtvBk2xvVAY9_4RTM-tU7gCIt6rWRh-Ok03kE5HtQWgevEOi1XOD-L9IgzsF_0g4DZ8U Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:06:01.242609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/f7d219e82efe44f9b6f860a5fabc7733"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_f7d219e82efe44f9b6f860a5fabc7733", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: f7d219e82efe44f9b6f860a5fabc7733 Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-811496748 Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 283a1bfe0e3b45aaa0a33e5fe6b1d85a Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-ServerRescueNegativeTestJSON-811496748-project Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 could not be found. Jul 24 01:06:01.244118 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '97d6538b-2585-40fc-b68c-e9a088724a39'} Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 could not be found. Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:01.244900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 could not be found. Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 could not be found. Jul 24 01:06:01.246024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:01.247325 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] Returning 404 to user: Instance 97d6538b-2585-40fc-b68c-e9a088724a39 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:01.247325 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-693c8e84-0ab5-4d1c-af48-c08294daff02 tempest-ServerRescueNegativeTestJSON-811496748 tempest-ServerRescueNegativeTestJSON-811496748-project] 10.222.0.18 "GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39" status: 404 len: 111 microversion: 2.1 time: 0.030955 Jul 24 01:06:01.247325 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 350/693] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:01 2022] GET /compute/v2.1/servers/97d6538b-2585-40fc-b68c-e9a088724a39 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:06:01.907942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e043d420-cfbc-4fb9-9026-40328d94c4bb"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:01.909068 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:01.917395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:01.918184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:01.918751 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:01.919234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:01.998507 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Invalid volume: volume e043d420-cfbc-4fb9-9026-40328d94c4bb is already attached to instances: 4693fbdb-0311-4f58-a0b7-e338e3369224 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: POST /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments HTTP/1.1 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 74 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:06:02.000265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {"volumeAttachment": {"volumeId": "e043d420-cfbc-4fb9-9026-40328d94c4bb"}} Jul 24 01:06:02.001808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Invalid volume: volume e043d420-cfbc-4fb9-9026-40328d94c4bb is already attached to instances: 4693fbdb-0311-4f58-a0b7-e338e3369224 Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'server_id': '4693fbdb-0311-4f58-a0b7-e338e3369224', 'body': {'volumeAttachment': {'volumeId': 'e043d420-cfbc-4fb9-9026-40328d94c4bb'}}} Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume e043d420-cfbc-4fb9-9026-40328d94c4bb is already attached to instances: 4693fbdb-0311-4f58-a0b7-e338e3369224 Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:02.002695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume e043d420-cfbc-4fb9-9026-40328d94c4bb is already attached to instances: 4693fbdb-0311-4f58-a0b7-e338e3369224 Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:02.004685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume e043d420-cfbc-4fb9-9026-40328d94c4bb is already attached to instances: 4693fbdb-0311-4f58-a0b7-e338e3369224 Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 400 to user: Invalid volume: volume e043d420-cfbc-4fb9-9026-40328d94c4bb is already attached to instances: 4693fbdb-0311-4f58-a0b7-e338e3369224 {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2f116dcb-7bd1-4559-9bf6-037216f9f60c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.098093 Jul 24 01:06:02.006229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 344/694] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:06:01 2022] POST /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments => generated 174 bytes in 99 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 24 01:06:02.063230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:02.063726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:02.070702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:02.071268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:02.072045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:02.072485 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:02.149576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=92 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:02.155495 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lazy-loading 'services' on Instance uuid 4693fbdb-0311-4f58-a0b7-e338e3369224 {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:02.234112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ff80189b-0b87-4d6f-91da-2e52b4bf5276 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments/e043d420-cfbc-4fb9-9026-40328d94c4bb" status: 202 len: 0 microversion: 2.1 time: 0.174577 Jul 24 01:06:02.235001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 351/695] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:06:02 2022] DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments/e043d420-cfbc-4fb9-9026-40328d94c4bb => generated 0 bytes in 176 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:06:02.244727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:02.245798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:02.253611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:02.254744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:02.255412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:02.255792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:02.291396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-93b508a5-870e-4082-a8ac-7257850377e6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050817 Jul 24 01:06:02.292167 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 345/696] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:02 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:03.306897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:03.308227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:03.330733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:03.331222 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:03.332154 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:03.332614 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:03.363101 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-85435683-47be-4c93-9d51-9d74ff0e739c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.061156 Jul 24 01:06:03.364288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 352/697] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:03 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224/os-volume_attachments => generated 25 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:06:03.427940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:03.428432 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:03.435080 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:03.435603 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:03.436434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:03.436953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:03.463308 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:06:03.542417 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-60a143ae-5ff7-4c9d-ac70-59835a1f13db tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 204 len: 0 microversion: 2.1 time: 0.118751 Jul 24 01:06:03.543219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 346/698] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:06:03 2022] DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:06:03.552240 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:03.552892 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:03.559139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:03.559795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:03.562003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:03.562646 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:03.716602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:03.726074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5982aab5-2ccd-4b93-80a6-a1f11ae0f89f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1681 microversion: 2.1 time: 0.178575 Jul 24 01:06:03.726931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 353/699] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:03 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1681 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:04.744465 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:04.745167 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:04.752221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:04.752676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:04.753460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:04.754578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:05.137574 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:05.148148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cccac35f-bc62-4ec4-a6c1-79357a6c2fda tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1681 microversion: 2.1 time: 0.407900 Jul 24 01:06:05.148719 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 347/700] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:04 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1681 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:06.166240 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:06.167517 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:06.176211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:06.176737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:06.177768 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:06.178268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:06.288947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:06.297939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2f594cc4-bae7-4133-8604-02559d999dc0 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 200 len: 1614 microversion: 2.1 time: 0.136198 Jul 24 01:06:06.298505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 354/701] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:06 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 1614 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:06.408422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-eae05583-15a0-4487-bbd4-538bb1607bb2 req-246e952a-2630-4486-81cb-e1f5dccf3533 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4693fbdb-0311-4f58-a0b7-e338e3369224", "name": "network-vif-deleted", "tag": "6545a60f-fbf2-432b-a011-f05ea4e793a1"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:06.418905 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-eae05583-15a0-4487-bbd4-538bb1607bb2 req-246e952a-2630-4486-81cb-e1f5dccf3533 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:06.419038 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-eae05583-15a0-4487-bbd4-538bb1607bb2 req-246e952a-2630-4486-81cb-e1f5dccf3533 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:06.436373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-eae05583-15a0-4487-bbd4-538bb1607bb2 req-246e952a-2630-4486-81cb-e1f5dccf3533 service nova] Creating event network-vif-deleted:6545a60f-fbf2-432b-a011-f05ea4e793a1 for instance 4693fbdb-0311-4f58-a0b7-e338e3369224 on hv-cinder-maste Jul 24 01:06:06.448122 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-eae05583-15a0-4487-bbd4-538bb1607bb2 req-246e952a-2630-4486-81cb-e1f5dccf3533 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043309 Jul 24 01:06:06.448880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 348/702] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:07.315005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:07.315615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:07.322509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:07.322987 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:07.323757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:07.324185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:07.340575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 HTTP/1.1 Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:06:07.342620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:06:07.344274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '4693fbdb-0311-4f58-a0b7-e338e3369224'} Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:07.345159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:06:07.346395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:07.348334 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:07.348334 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-71b2829d-8617-47d9-8b26-a8c30440f22e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 404 len: 111 microversion: 2.1 time: 0.033524 Jul 24 01:06:07.348334 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 355/703] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:07 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:06:07.356094 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e99d78db-97dd-4fa8-bfd3-d7858db8c85a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '.version_select at 0x7f5e3035e9d0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:07.374732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e99d78db-97dd-4fa8-bfd3-d7858db8c85a tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1444987007" status: 202 len: 0 microversion: 2.1 time: 0.022823 Jul 24 01:06:07.375365 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 349/704] 10.222.0.18 () {58 vars in 1220 bytes} [Sun Jul 24 01:06:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1444987007 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:06:08.119053 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:08.119989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:08.129076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:08.129767 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:08.131342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:08.131931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:08.156681 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f2abd5ed-acef-4beb-aa78-19a314c83a60 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/ips" status: 200 len: 151 microversion: 2.20 time: 0.256699 Jul 24 01:06:08.157275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 356/705] 10.222.0.18 () {60 vars in 1288 bytes} [Sun Jul 24 01:06:07 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/ips => generated 151 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:06:08.206569 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-eb1deafa-d74d-4759-9d1e-d593ffaba0ad tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: .version_select at 0x7f5e302b7280>, body: {"keypair": {"name": "tempest-keypair-1567354850", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCjleh1bLSA+VnMyZCSSuh28jC6zWqM17XJLomHryKX7mzoyVXF6H8/VEpxfw39RIdj/fIOBaEGrYKNHxwvXxNMeZ0wS36B0Pb5sPS2zPB89mQbalITotLIi7hYxCvPYRg=="}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:08.210548 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=4,nova_api:DELETE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:08.237006 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-eb1deafa-d74d-4759-9d1e-d593ffaba0ad tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:06:08.241958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-eb1deafa-d74d-4759-9d1e-d593ffaba0ad tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:06:08.266787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-eb1deafa-d74d-4759-9d1e-d593ffaba0ad tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.064644 Jul 24 01:06:08.267497 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 350/706] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:06:08 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:09.646613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-186351028", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "37cd1572-14d0-471e-b2ca-ff3f9078882e"}], "security_groups": [{"name": "tempest-securitygroup--516818891"}], "key_name": "tempest-keypair-1567354850", "user_data": "}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:09.653724 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=47,nova_api:DELETE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:09.756654 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] validate_networks() for [('37cd1572-14d0-471e-b2ca-ff3f9078882e', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:06:10.303453 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:06:10.304583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:06:10.304583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:06:10.304818 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:06:10.305174 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:06:10.315899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:10.316495 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:10.317347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:10.317785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:10.336138 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:06:10.340469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Getting quotas for user 8f471531dac249d994f864102f7b9d3f and project 3c68e39bc1724fda980be3463bbf7d7b. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:06:10.353086 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:06:10.356524 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:06:10.415414 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1ba3ea2a-28e4-4cb5-8e55-be29087d1f76 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.773091 Jul 24 01:06:10.416064 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 357/707] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:06:09 2022] POST /compute/v2.1/servers => generated 401 bytes in 775 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:06:10.427517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2c7a4e79-6512-4243-bd14-e9bf816b445c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:10.429035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2c7a4e79-6512-4243-bd14-e9bf816b445c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:10.500826 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2c7a4e79-6512-4243-bd14-e9bf816b445c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:10.510511 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2c7a4e79-6512-4243-bd14-e9bf816b445c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1241 microversion: 2.1 time: 0.086755 Jul 24 01:06:10.511353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 351/708] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:10 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1241 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:10.657092 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:10.657757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:10.671610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:10.672276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:10.673292 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:10.673993 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:10.704622 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lazy-loading 'flavor' on Instance uuid c70b3837-80be-480a-9a3f-7b03aff78573 {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:11.046522 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-54c47ee8-2b32-46f9-8ce8-0975155a578a tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/action" status: 202 len: 0 microversion: 2.20 time: 0.395742 Jul 24 01:06:11.050182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 358/709] 10.222.0.18 () {62 vars in 1318 bytes} [Sun Jul 24 01:06:10 2022] POST /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/action => generated 0 bytes in 398 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:06:11.060498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:11.061623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:11.070849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:11.071561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:11.072610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:11.073210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:11.267634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:11.276914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:11.278617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:11.279953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1ade6e85-5a2f-4edf-aead-f3db68aa7584 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.227152 Jul 24 01:06:11.280532 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 352/710] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:11 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:11.530681 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:11.531416 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:11.539088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:11.539543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:11.540315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:11.540751 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:11.607233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:11.614804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-787d3a2c-fc71-46bc-9b79-a7fa280b9870 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1293 microversion: 2.1 time: 0.089928 Jul 24 01:06:11.615365 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 359/711] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:11 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1293 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:12.300487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:12.301676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:12.308794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:12.309492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:12.310505 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:12.310983 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:12.504169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:12.512918 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:12.514441 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:12.515863 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a523f436-a8c8-429f-8899-7e7141617e82 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.220063 Jul 24 01:06:12.516426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 353/712] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:12 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:12.636124 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:12.636719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:12.643016 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:12.643474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:12.644245 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:12.644855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:12.890006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:12.898261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-24c946a6-dcd2-42e8-ad44-2d929d0b787c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.270111 Jul 24 01:06:12.898861 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 360/713] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:12 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:13.539970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:13.540587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:13.553206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:13.557785 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:13.558640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:13.560633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:13.829122 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:13.852820 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:13.854412 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:13.860571 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d2afdcee-e155-4def-bce6-cccf50b5108b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.325859 Jul 24 01:06:13.861495 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 354/714] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:13 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:13.916242 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:13.917139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:13.925626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:13.926744 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:13.928355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:13.929088 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:13.988415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-3d359039-f134-46e8-8bc7-cd2470ebf903 req-7497cdf2-7776-4734-b099-6fd716dc8aed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e985f6e0-7865-4447-868d-568639afe87b", "tag": "87ad2158-30f6-4543-a57f-4f732fe5c551"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:13.998559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3d359039-f134-46e8-8bc7-cd2470ebf903 req-7497cdf2-7776-4734-b099-6fd716dc8aed service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:13.999624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-3d359039-f134-46e8-8bc7-cd2470ebf903 req-7497cdf2-7776-4734-b099-6fd716dc8aed service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:14.014610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-3d359039-f134-46e8-8bc7-cd2470ebf903 req-7497cdf2-7776-4734-b099-6fd716dc8aed service nova] Creating event network-changed:87ad2158-30f6-4543-a57f-4f732fe5c551 for instance e985f6e0-7865-4447-868d-568639afe87b on hv-cinder-maste Jul 24 01:06:14.020574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-3d359039-f134-46e8-8bc7-cd2470ebf903 req-7497cdf2-7776-4734-b099-6fd716dc8aed service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.225118 Jul 24 01:06:14.022592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 361/715] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:14.158563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:14.166798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d1e6b3ee-1081-4af5-91a7-50478b0b2961 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.255735 Jul 24 01:06:14.167446 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 355/716] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:13 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:14.881077 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:14.881739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:14.888041 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:14.888479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:14.889258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:14.889685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:15.017134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:15.026290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:15.027802 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:15.028941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-39bd52e3-4356-458c-a8a2-260da9374f69 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.153285 Jul 24 01:06:15.029518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 362/717] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:14 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:15.183977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:15.184722 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:15.191632 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:15.192036 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:15.193882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:15.194681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:15.378136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:15.389247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bc055051-5245-4a41-bf45-99abc1cea28e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.208382 Jul 24 01:06:15.390058 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 356/718] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:15 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:16.046516 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:16.047093 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:16.054162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.054610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.055415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.055848 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.139489 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1372850725"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:16.140020 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:16.146828 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.147193 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.147975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.148406 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.507889 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:16.515798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:16.517344 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:16.518523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fc83459d-9c52-4165-950b-5c5f6f8c9ee6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.476015 Jul 24 01:06:16.522828 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 363/719] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:16 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:16.523121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.network.security_group_api [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Removing security group b8b34d0f-b8f7-4710-9499-8209c663a9ff from port 83a88c5b-6010-4bfd-9dd1-ee05588aab25 Jul 24 01:06:16.523554 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:16.524125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:16.529958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.530396 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.531193 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.531620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.745462 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:16.754168 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a5168f79-9e4f-4360-a2bb-dd167333a0b1 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.234095 Jul 24 01:06:16.754869 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 364/720] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:16 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:16.954050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1ce116bf-07c2-4288-9080-43ea91e8d859 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/action" status: 202 len: 0 microversion: 2.1 time: 0.818109 Jul 24 01:06:16.955957 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 357/721] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:06:16 2022] POST /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/action => generated 0 bytes in 821 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:06:16.966517 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-a6d8d99d-fdb1-4282-bc3f-7bfe5ccf2763 req-9b99a7b1-088f-4c3b-9876-20c2b91ca741 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:16.977180 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-a6d8d99d-fdb1-4282-bc3f-7bfe5ccf2763 req-9b99a7b1-088f-4c3b-9876-20c2b91ca741 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:16.979782 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-a6d8d99d-fdb1-4282-bc3f-7bfe5ccf2763 req-9b99a7b1-088f-4c3b-9876-20c2b91ca741 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:16.993702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-a6d8d99d-fdb1-4282-bc3f-7bfe5ccf2763 req-9b99a7b1-088f-4c3b-9876-20c2b91ca741 service nova] Creating event network-changed:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:06:17.000278 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-a6d8d99d-fdb1-4282-bc3f-7bfe5ccf2763 req-9b99a7b1-088f-4c3b-9876-20c2b91ca741 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037609 Jul 24 01:06:17.001439 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 365/722] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:17.537670 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:17.539926 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:17.556759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:17.557210 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:17.557990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:17.558428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:17.775236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:17.776375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:17.812037 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:17.813235 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:17.814099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:17.814542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:17.963223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:17.979144 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:17.981949 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:17.984289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5850b41e-9e5c-4d7c-8f16-cc6c0ebee39d tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.451084 Jul 24 01:06:17.985487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 358/723] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:17 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:18.140547 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:18.153547 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7714b8e9-684f-4897-a942-266cdb23686c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.385350 Jul 24 01:06:18.154544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 366/724] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:17 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:06:18.219710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:18.220289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:18.229255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:18.230075 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:18.231578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:18.232406 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:18.269765 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8b505061-5ded-4763-a17d-417b80a08dfe tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/ips" status: 200 len: 145 microversion: 2.1 time: 0.282964 Jul 24 01:06:18.270331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 359/725] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:06:17 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/ips => generated 145 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:18.755499 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:18.755995 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:18.762125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:18.762575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:18.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:18.763807 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:18.863320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lazy-loading 'services' on Instance uuid 12ae94be-ce19-4874-8fda-67767118d13f {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:18.947321 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-898f8ab1-cdf0-40fc-baa8-fee3e6c6a0dc tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments/ec0df126-d8c4-43b0-8666-778bb2276d7e" status: 202 len: 0 microversion: 2.1 time: 0.195497 Jul 24 01:06:18.948312 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 367/726] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:06:18 2022] DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments/ec0df126-d8c4-43b0-8666-778bb2276d7e => generated 0 bytes in 197 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:06:18.956709 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:18.958902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:18.966038 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:18.966969 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:18.968031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:18.968477 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.004003 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:19.004721 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:19.011349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.011801 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.012567 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.013000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.017258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:19.023406 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3786ce93-ae1a-448d-80a5-f0237b5c2895 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070638 Jul 24 01:06:19.023986 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 360/727] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:18 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:19.029250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-9b41f2ce-36de-4c38-80cc-45905ff97539 req-a0a6a908-5144-48f5-a801-2201fd7cd3a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:19.040257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-9b41f2ce-36de-4c38-80cc-45905ff97539 req-a0a6a908-5144-48f5-a801-2201fd7cd3a2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.040850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-9b41f2ce-36de-4c38-80cc-45905ff97539 req-a0a6a908-5144-48f5-a801-2201fd7cd3a2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.054265 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-9b41f2ce-36de-4c38-80cc-45905ff97539 req-a0a6a908-5144-48f5-a801-2201fd7cd3a2 service nova] Creating event network-changed:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:06:19.060814 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-9b41f2ce-36de-4c38-80cc-45905ff97539 req-a0a6a908-5144-48f5-a801-2201fd7cd3a2 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035428 Jul 24 01:06:19.061726 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 361/728] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:19.173648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:19.175487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:19.183712 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:19.185048 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.185687 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.186753 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.187428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.195634 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:19.198005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:19.199233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-51b19535-2597-4164-9ff0-1d2c69cdad56 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.199577 Jul 24 01:06:19.202815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 368/729] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:18 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:19.341682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:19.349740 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ca5149dd-474f-4026-82a5-b7fe68da610c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.182049 Jul 24 01:06:19.359151 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 362/730] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:19 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:19.709668 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "07cc3376-583c-4c84-8384-7ed41168df04"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:19.710951 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:19.717320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.717839 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.718834 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.719272 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:19.975462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-d19d7936-79b4-4f12-8ac6-27047130dd50 req-d3e3526f-9d53-4dde-8cb6-fea8172a0fe2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e985f6e0-7865-4447-868d-568639afe87b", "name": "network-vif-plugged", "status": "completed", "tag": "87ad2158-30f6-4543-a57f-4f732fe5c551"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:19.985476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-d19d7936-79b4-4f12-8ac6-27047130dd50 req-d3e3526f-9d53-4dde-8cb6-fea8172a0fe2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:19.986239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-d19d7936-79b4-4f12-8ac6-27047130dd50 req-d3e3526f-9d53-4dde-8cb6-fea8172a0fe2 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.025795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-d19d7936-79b4-4f12-8ac6-27047130dd50 req-d3e3526f-9d53-4dde-8cb6-fea8172a0fe2 service nova] Creating event network-vif-plugged:87ad2158-30f6-4543-a57f-4f732fe5c551 for instance e985f6e0-7865-4447-868d-568639afe87b on hv-cinder-maste Jul 24 01:06:20.035184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-d19d7936-79b4-4f12-8ac6-27047130dd50 req-d3e3526f-9d53-4dde-8cb6-fea8172a0fe2 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.300111 Jul 24 01:06:20.035802 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 363/731] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:06:20.050400 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:20.052277 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:20.064029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.064677 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.065515 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.065942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.084447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:20.105610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a357d8a3-a697-4a74-9b34-e5feb0bb3dc7 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.067811 Jul 24 01:06:20.106542 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 364/732] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:20 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments => generated 25 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:06:20.123025 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:20.123522 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:20.136933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.137552 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.138561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.139156 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.158630 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-619a2846-8ed9-4408-aae0-68c24406994c tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.453152 Jul 24 01:06:20.159916 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 369/733] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:06:19 2022] POST /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments => generated 194 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:20.219951 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:20.220955 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:20.240502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.243677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.245492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.246574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.282963 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] HTTP exception thrown: Instance 12ae94be-ce19-4874-8fda-67767118d13f is not attached to volume ec0df126-d8c4-43b0-8666-778bb2276d7e Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Request method failure captured: Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments/ec0df126-d8c4-43b0-8666-778bb2276d7e HTTP/1.1 Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jpbp2_C4G8Av6-87RE3u_ZM8XUTxzmHYyVxw8EXd794uZguDrxclf694-rG2X8wU7UwbaQumrXFNBo9uLVruYwtdGywplmgHljsUsNyX5JoX2slukpuG2y0d4Abv0h1VOhynkdZIsl2XfLc53Oapzb8N16l2Y3v2VOyOGl1VS9bzPo Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: ee9d5b852be444f48a33b2f4be5c34be Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-TestMinimumBasicScenario-1888617681 Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:06:20.285383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/ee9d5b852be444f48a33b2f4be5c34be"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/ee9d5b852be444f48a33b2f4be5c34be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/ee9d5b852be444f48a33b2f4be5c34be"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_ee9d5b852be444f48a33b2f4be5c34be", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-TestMinimumBasicScenario-1888617681 Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: ee9d5b852be444f48a33b2f4be5c34be Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1888617681 Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-TestMinimumBasicScenario-1888617681-project Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: ddda2bac031d4635b27421d22bb40fff Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-TestMinimumBasicScenario-1888617681-project Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:06:20.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 12ae94be-ce19-4874-8fda-67767118d13f is not attached to volume ec0df126-d8c4-43b0-8666-778bb2276d7e Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'server_id': '12ae94be-ce19-4874-8fda-67767118d13f', 'id': 'ec0df126-d8c4-43b0-8666-778bb2276d7e'} Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id ec0df126-d8c4-43b0-8666-778bb2276d7e. Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:20.288533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 12ae94be-ce19-4874-8fda-67767118d13f is not attached to volume ec0df126-d8c4-43b0-8666-778bb2276d7e Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 12ae94be-ce19-4874-8fda-67767118d13f is not attached to volume ec0df126-d8c4-43b0-8666-778bb2276d7e Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:20.289711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Returning 404 to user: Instance 12ae94be-ce19-4874-8fda-67767118d13f is not attached to volume ec0df126-d8c4-43b0-8666-778bb2276d7e {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:20.299889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-22b2f20f-1138-44a1-b2d0-fe5f5568ab3f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments/ec0df126-d8c4-43b0-8666-778bb2276d7e" status: 404 len: 154 microversion: 2.1 time: 0.181348 Jul 24 01:06:20.300476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 365/734] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:06:20 2022] DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f/os-volume_attachments/ec0df126-d8c4-43b0-8666-778bb2276d7e => generated 154 bytes in 184 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:06:20.326746 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:20.347983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:20.368964 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:20.369976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:20.378954 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.379721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.380833 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:20.381514 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:20.506649 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:20.551908 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:20.562378 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:20.564420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:20.566161 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5d29a2fe-faa8-433a-be38-9fd9b5ffcb74 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.352040 Jul 24 01:06:20.566909 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 370/735] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:20 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:20.592970 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:20.601143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8b93374e-9848-4c58-aa3a-605ae542bfb3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.237311 Jul 24 01:06:20.601794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 366/736] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:20 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:06:21.497672 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:21.498169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:21.506453 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.506730 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.507534 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.507970 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.533763 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:06:21.585790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:21.586472 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:21.594058 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.594518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.595561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.596150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.613536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-da4d0ac9-7830-4e9b-8449-3225b39f800d tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 204 len: 0 microversion: 2.1 time: 0.119354 Jul 24 01:06:21.615442 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 371/737] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:06:21 2022] DELETE /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:06:21.624736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:21.625681 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:21.632457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.633004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.633783 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.634242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.796209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:21.813060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:21.814934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:21.816403 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6bb80593-8c90-402a-94cb-9d11882855cc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.235020 Jul 24 01:06:21.827379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:21.829100 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 367/738] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:21 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:21.836138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:21.836784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9e71a47a-7af9-445d-bb8e-5f96d47cdc79 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1366 microversion: 2.1 time: 0.216780 Jul 24 01:06:21.837311 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:21.837435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 372/739] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:21 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1366 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:21.843789 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.844424 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.845347 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:21.845886 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:21.993180 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:22.001429 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1cf9032c-470c-4672-9ecd-b5ac0b00559c tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1555 microversion: 2.1 time: 0.170995 Jul 24 01:06:22.013588 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 368/740] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:21 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1555 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:22.835230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:22.835835 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:22.842264 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:22.842719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:22.843516 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:22.843954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:22.853430 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:22.854137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:22.860641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:22.861085 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:22.861871 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:22.862304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:23.033759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:23.042369 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:23.043927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:23.045088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c2ebc686-14c0-4323-aa16-99134253e70e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1743 microversion: 2.20 time: 0.213864 Jul 24 01:06:23.045521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:23.046034 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 373/741] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:22 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1743 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:23.050621 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:23.051256 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:23.054975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-afc353f8-d27d-4e20-be19-cc8bb63b410f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1556 microversion: 2.1 time: 0.205314 Jul 24 01:06:23.055815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 369/742] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:22 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1556 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:06:23.058541 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:23.059029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:23.059808 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:23.060247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:23.071929 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:23.072424 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:23.078429 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:23.078903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:23.079678 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:23.080135 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:23.188682 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f6bf57a0-f61e-490b-b38e-5e1872e8c390 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.121022 Jul 24 01:06:23.189282 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 370/743] 10.222.0.18 () {58 vars in 1274 bytes} [Sun Jul 24 01:06:23 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-interface => generated 284 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:06:23.234208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:23.241942 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-087e97fa-137d-4c1e-a7d8-70f347abd8f3 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1555 microversion: 2.1 time: 0.194759 Jul 24 01:06:23.242579 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 374/744] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:23 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1555 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:24.064933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:24.065546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:24.072526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:24.072978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:24.073750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:24.074185 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:24.214394 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:24.223172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:24.225479 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:24.226659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c547f978-773e-4029-8f65-2e26976b8d04 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1743 microversion: 2.20 time: 0.166161 Jul 24 01:06:24.237999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 371/745] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:24 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1743 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:24.263302 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:24.264198 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:24.272888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:24.273349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:24.274218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:24.274662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:24.298469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-44305561-d460-4c71-baf2-b6802128f7af req-eb219b81-9466-4289-8940-2cd005c26c3b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e985f6e0-7865-4447-868d-568639afe87b", "tag": "87ad2158-30f6-4543-a57f-4f732fe5c551"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:24.306855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-44305561-d460-4c71-baf2-b6802128f7af req-eb219b81-9466-4289-8940-2cd005c26c3b service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:24.307514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-44305561-d460-4c71-baf2-b6802128f7af req-eb219b81-9466-4289-8940-2cd005c26c3b service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:24.319535 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-44305561-d460-4c71-baf2-b6802128f7af req-eb219b81-9466-4289-8940-2cd005c26c3b service nova] Creating event network-changed:87ad2158-30f6-4543-a57f-4f732fe5c551 for instance e985f6e0-7865-4447-868d-568639afe87b on hv-cinder-maste Jul 24 01:06:24.325148 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-44305561-d460-4c71-baf2-b6802128f7af req-eb219b81-9466-4289-8940-2cd005c26c3b service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.173820 Jul 24 01:06:24.325398 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 375/746] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:06:24.420207 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:24.428047 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e7b619d3-65c3-42a6-8ec2-fc5e816877c5 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1555 microversion: 2.1 time: 0.170470 Jul 24 01:06:24.428620 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 372/747] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:24 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1555 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:25.252499 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:25.254172 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:25.265323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.267011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.268221 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.268937 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.446382 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:25.447617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:25.455714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.456601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.457620 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.458884 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.485733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:25.494258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:25.494457 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.495026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.495644 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lazy-loading 'fault' on Instance uuid 12ae94be-ce19-4874-8fda-67767118d13f {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:25.496216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:25.497900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-44118183-df19-491e-a113-dfb30873eab2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1743 microversion: 2.20 time: 0.255545 Jul 24 01:06:25.499027 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 376/748] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:25 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1743 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:25.549463 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ddda2bac031d4635b27421d22bb40fff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee9d5b852be444f48a33b2f4be5c34be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:25.559953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7827270d-d0b8-4c5c-aaee-39c215a250bb tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 200 len: 1349 microversion: 2.1 time: 0.118208 Jul 24 01:06:25.561102 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 373/749] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:25 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 1349 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:06:25.636946 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:25.638321 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:25.644761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.645986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.646978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.647571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.680239 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-033a665e-0b0c-4037-92b4-4928da9a4584 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047231 Jul 24 01:06:25.685444 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 377/750] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:25 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:25.691427 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:25.692422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:25.701334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.701788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.702709 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.703188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.736051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-78e8c4cd-320c-46b0-9408-ffa112be4b37 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/07cc3376-583c-4c84-8384-7ed41168df04" status: 200 len: 194 microversion: 2.1 time: 0.049214 Jul 24 01:06:25.736683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 374/751] 10.222.0.18 () {58 vars in 1412 bytes} [Sun Jul 24 01:06:25 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/07cc3376-583c-4c84-8384-7ed41168df04 => generated 194 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:06:25.769087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-2a2623d7-d0dd-40ce-abb0-e5ab889e072a req-5ebb0812-5e30-4d2e-b19c-e2a868789fc6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "name": "network-vif-plugged", "status": "completed", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:25.779991 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2a2623d7-d0dd-40ce-abb0-e5ab889e072a req-5ebb0812-5e30-4d2e-b19c-e2a868789fc6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:25.780630 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2a2623d7-d0dd-40ce-abb0-e5ab889e072a req-5ebb0812-5e30-4d2e-b19c-e2a868789fc6 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:25.795804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-2a2623d7-d0dd-40ce-abb0-e5ab889e072a req-5ebb0812-5e30-4d2e-b19c-e2a868789fc6 service nova] Creating event network-vif-plugged:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:06:25.801480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-2a2623d7-d0dd-40ce-abb0-e5ab889e072a req-5ebb0812-5e30-4d2e-b19c-e2a868789fc6 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037900 Jul 24 01:06:25.802757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 378/752] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:06:26.339291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-52aa9604-3a8a-4d18-99b2-e0ac67f03760 req-02f7c69a-cc17-43c1-986a-776fde51aea7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "12ae94be-ce19-4874-8fda-67767118d13f", "name": "network-vif-deleted", "tag": "83a88c5b-6010-4bfd-9dd1-ee05588aab25"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:26.347487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-52aa9604-3a8a-4d18-99b2-e0ac67f03760 req-02f7c69a-cc17-43c1-986a-776fde51aea7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:26.347932 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-52aa9604-3a8a-4d18-99b2-e0ac67f03760 req-02f7c69a-cc17-43c1-986a-776fde51aea7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:26.359660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-52aa9604-3a8a-4d18-99b2-e0ac67f03760 req-02f7c69a-cc17-43c1-986a-776fde51aea7 service nova] Creating event network-vif-deleted:83a88c5b-6010-4bfd-9dd1-ee05588aab25 for instance 12ae94be-ce19-4874-8fda-67767118d13f on hv-cinder-maste Jul 24 01:06:26.364809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-52aa9604-3a8a-4d18-99b2-e0ac67f03760 req-02f7c69a-cc17-43c1-986a-776fde51aea7 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028839 Jul 24 01:06:26.365407 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 375/753] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:26.518486 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:26.519212 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:26.526977 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:26.527604 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:26.528367 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:26.528796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:26.579079 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:26.580444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] [instance: 12ae94be-ce19-4874-8fda-67767118d13f] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:26.588783 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:26.589552 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:26.590413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:26.591085 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:26.607236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] HTTP exception thrown: Instance 12ae94be-ce19-4874-8fda-67767118d13f could not be found. Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Request method failure captured: Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f HTTP/1.1 Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jpbp2_C4G8Av6-87RE3u_ZM8XUTxzmHYyVxw8EXd794uZguDrxclf694-rG2X8wU7UwbaQumrXFNBo9uLVruYwtdGywplmgHljsUsNyX5JoX2slukpuG2y0d4Abv0h1VOhynkdZIsl2XfLc53Oapzb8N16l2Y3v2VOyOGl1VS9bzPo Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: ee9d5b852be444f48a33b2f4be5c34be Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-TestMinimumBasicScenario-1888617681 Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:06:26.609315 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/ee9d5b852be444f48a33b2f4be5c34be"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/ee9d5b852be444f48a33b2f4be5c34be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/ee9d5b852be444f48a33b2f4be5c34be"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_ee9d5b852be444f48a33b2f4be5c34be", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-TestMinimumBasicScenario-1888617681 Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: ee9d5b852be444f48a33b2f4be5c34be Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1888617681 Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-TestMinimumBasicScenario-1888617681-project Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: ddda2bac031d4635b27421d22bb40fff Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-TestMinimumBasicScenario-1888617681-project Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 12ae94be-ce19-4874-8fda-67767118d13f could not be found. Jul 24 01:06:26.610750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '12ae94be-ce19-4874-8fda-67767118d13f'} Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 12ae94be-ce19-4874-8fda-67767118d13f could not be found. Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:26.611530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 12ae94be-ce19-4874-8fda-67767118d13f could not be found. Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 12ae94be-ce19-4874-8fda-67767118d13f could not be found. Jul 24 01:06:26.612729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:26.617560 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Returning 404 to user: Instance 12ae94be-ce19-4874-8fda-67767118d13f could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:26.617560 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b4b13ac8-074f-441d-a12d-98395f0d470f tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f" status: 404 len: 111 microversion: 2.1 time: 0.037257 Jul 24 01:06:26.618614 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 376/754] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:26 2022] GET /compute/v2.1/servers/12ae94be-ce19-4874-8fda-67767118d13f => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:06:26.624788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-538c9bfd-c9ba-4d79-99ff-9e9118502e14 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] Calling method '.version_select at 0x7f5e3033eee0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:26.649838 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-538c9bfd-c9ba-4d79-99ff-9e9118502e14 tempest-TestMinimumBasicScenario-1888617681 tempest-TestMinimumBasicScenario-1888617681-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1269427714" status: 202 len: 0 microversion: 2.1 time: 0.029664 Jul 24 01:06:26.650880 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 377/755] 10.222.0.18 () {58 vars in 1271 bytes} [Sun Jul 24 01:06:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1269427714 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:06:26.693554 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:26.705075 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:26.707362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:26.709312 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3d0ef3f2-a109-4051-acd7-167a92626c80 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.195965 Jul 24 01:06:26.710014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 379/756] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:26 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:27.113211 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5820a72d-310e-4b12-b7d2-dd94d19dbc91"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:27.114322 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:27.122373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:27.122712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:27.123757 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:27.124196 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:27.486562 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-386cc404-63a4-44b4-b1c5-184425b53f81 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "POST /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.377043 Jul 24 01:06:27.488098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 378/757] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:06:27 2022] POST /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments => generated 194 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:27.732082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:27.732691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:27.743240 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:27.747330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:27.748393 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:27.749069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:27.982318 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:27.997339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:28.004984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:28.006396 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-032b23f9-1a23-49fe-bc93-f693e3ecb2b3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.278982 Jul 24 01:06:28.007391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 380/758] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:27 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:29.026711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:29.027499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:29.035288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:29.036413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:29.037314 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:29.037859 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:29.190147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:29.213274 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:29.215346 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:29.219263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b9e318f9-f32b-441c-b420-a6b0f948cf1f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.197523 Jul 24 01:06:29.252255 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 379/759] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:29 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:30.244446 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:30.245105 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:30.252065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:30.252718 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:30.253821 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:30.254260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:30.602262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:30.619819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:30.632042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:30.633533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9f069bae-5743-4f1d-bded-c41dd3ed64fe tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.393144 Jul 24 01:06:30.634606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 381/760] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:30 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:31.654160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:31.655601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:31.662375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:31.670227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:31.678781 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:31.679336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:31.973663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:31.983401 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:31.985167 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:31.986905 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-796f0875-e4c0-456e-99a6-e8354567541c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.338351 Jul 24 01:06:31.987659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 380/761] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:31 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:33.007395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:33.008012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:33.016627 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:33.017640 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:33.018742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:33.019419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:33.242591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:33.256171 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:33.258507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:33.273821 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7f15ce91-e5b4-437a-8598-786b667e2a6f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.262507 Jul 24 01:06:33.274856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 382/762] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:33 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:34.298653 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:34.298653 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:34.313973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:34.314527 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:34.317435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:34.317934 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:34.577990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:34.605378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:34.615621 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:34.617101 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d24d9c98-c516-4ff0-a0cd-42f1510094e0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.327546 Jul 24 01:06:34.618042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 381/763] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:34 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:35.328275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:35.329862 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:35.345873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.346245 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.347555 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.350551 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.384561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:35.399883 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-02d39208-687e-430d-b41d-57cf9245e56e tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.076443 Jul 24 01:06:35.405542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 383/764] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:35 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments => generated 371 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:35.616553 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:35.617045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:35.626673 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.627145 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.628101 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.630669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.639559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:35.640572 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:35.653634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:35.655296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.657425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.658395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.659687 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.741089 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-71b3664c-e959-4339-b8f8-f1856da5f9b5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/07cc3376-583c-4c84-8384-7ed41168df04" status: 200 len: 194 microversion: 2.1 time: 0.129094 Jul 24 01:06:35.742118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 382/765] 10.222.0.18 () {58 vars in 1412 bytes} [Sun Jul 24 01:06:35 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/07cc3376-583c-4c84-8384-7ed41168df04 => generated 194 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:35.758292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:35.759749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:35.779339 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.780179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.781831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:35.782547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:35.921658 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lazy-loading 'services' on Instance uuid 1c4a54e8-8220-4b56-bbb7-51402654c743 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:36.024627 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:36.037954 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:36.042042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:36.050794 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2549eae3-481d-45d6-856f-e5c6801549bc tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.418929 Jul 24 01:06:36.053715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 384/766] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:35 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:36.101156 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0724fbd9-8bbc-4e70-989b-83d76bdf1ecd tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/07cc3376-583c-4c84-8384-7ed41168df04" status: 202 len: 0 microversion: 2.1 time: 0.349913 Jul 24 01:06:36.103317 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 383/767] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:06:35 2022] DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/07cc3376-583c-4c84-8384-7ed41168df04 => generated 0 bytes in 353 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:06:37.072674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:37.074219 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:37.085719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:37.086178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:37.087039 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:37.087477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:37.395276 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:37.408867 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:37.424678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:37.431293 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-96628a94-aa0e-4eaa-ab20-395646ed00f4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.361622 Jul 24 01:06:37.431998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 385/768] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:37 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:38.456610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:38.456610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:38.462491 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:38.464119 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:38.473800 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:38.474810 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:38.476799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:38.477593 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:38.478688 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:38.479392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:38.480426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:38.481117 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:38.485972 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:38.552911 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-45123702-f69c-44e5-a815-f7514e16d1b2 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/5820a72d-310e-4b12-b7d2-dd94d19dbc91" status: 200 len: 194 microversion: 2.1 time: 0.094498 Jul 24 01:06:38.554049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 386/769] 10.222.0.18 () {58 vars in 1412 bytes} [Sun Jul 24 01:06:38 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/5820a72d-310e-4b12-b7d2-dd94d19dbc91 => generated 194 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:38.567367 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:38.571898 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:38.610217 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:38.611480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:38.617660 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:38.618400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:38.770345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:38.778741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lazy-loading 'services' on Instance uuid 1c4a54e8-8220-4b56-bbb7-51402654c743 {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:38.780141 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:38.781669 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:38.783063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1a37f3c4-f448-4842-ab13-793139584979 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1738 microversion: 2.20 time: 0.336804 Jul 24 01:06:38.783440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 384/770] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:38 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1738 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:38.872659 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-bd5047f1-7c18-4c61-85e8-a9ea4778efd6 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/5820a72d-310e-4b12-b7d2-dd94d19dbc91" status: 202 len: 0 microversion: 2.1 time: 0.310513 Jul 24 01:06:38.873557 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 387/771] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:06:38 2022] DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments/5820a72d-310e-4b12-b7d2-dd94d19dbc91 => generated 0 bytes in 312 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:06:39.802021 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:39.802719 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:39.811053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:39.811557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:39.812609 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:39.813264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:39.953073 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:39.963099 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:39.965150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:39.966334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-100cb321-908a-421f-af80-03999e6d056b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1720 microversion: 2.20 time: 0.169371 Jul 24 01:06:39.967154 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 385/772] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:39 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1720 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:40.986908 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:40.987680 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:40.989827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:40.995793 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:40.996494 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:40.997487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:40.997929 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.136780 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:41.146330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:41.147505 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:41.153899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:41.154199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.154659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.155472 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.155903 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:41.156069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.157506 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-50b0d9f0-1f7f-4b0a-9e87-be6f61ee7e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1720 microversion: 2.20 time: 0.175539 Jul 24 01:06:41.158578 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 388/773] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:40 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1720 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:41.195010 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1ec013ab-4ddc-4c2d-bed6-9b0f2c9ec874 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053133 Jul 24 01:06:41.195889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 386/774] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:41 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 24 01:06:41.310826 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:41.311815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:41.319270 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.319717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.320482 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.320914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.350608 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3f686701-e8c4-45b2-8350-f797299d7467 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043995 Jul 24 01:06:41.351190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 389/775] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:06:41 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:06:41.409549 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:41.410068 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:41.416282 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.416754 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.417525 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.417966 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.445580 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:06:41.518196 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d082874f-122a-49c1-8e20-6a2040c76636 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 204 len: 0 microversion: 2.1 time: 0.112116 Jul 24 01:06:41.518787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 387/776] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:06:41 2022] DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:06:41.527085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:41.527693 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:41.533780 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.534227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.535012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:41.535460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:41.661856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:41.669371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b7d7416b-a54b-438f-84ef-3f14d84621db tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1672 microversion: 2.1 time: 0.146280 Jul 24 01:06:41.669936 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 390/777] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:41 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1672 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:42.177657 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:42.178345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:42.184923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:42.185372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:42.186135 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:42.186575 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:42.319335 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:42.327589 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:42.329086 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:42.330238 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1e67f8b8-fafc-4716-bfb4-0cf548200e32 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1720 microversion: 2.20 time: 0.157531 Jul 24 01:06:42.330816 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 388/778] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:42 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1720 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:42.686528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:42.687165 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:42.694096 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:42.694541 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:42.695326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:42.695760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:42.821425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:42.829302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-cd7f2de4-aba7-46ab-bdc1-f8487386f8ee tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1672 microversion: 2.1 time: 0.146634 Jul 24 01:06:42.829863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 391/779] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:42 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1672 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:43.349931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:43.350526 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:43.357216 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:43.357681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:43.358448 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:43.358933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:43.488777 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:43.497517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:43.499700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:43.500861 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6f1c8994-0a3a-4fa9-90e4-e5ec82c5ff5e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1720 microversion: 2.20 time: 0.155472 Jul 24 01:06:43.501595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 389/780] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:43 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1720 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:43.847169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:43.847709 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:43.854748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:43.855226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:43.855990 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:43.856424 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:43.983457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:43.990989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-67807494-1d18-4da4-97b5-77b43c279659 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1672 microversion: 2.1 time: 0.147758 Jul 24 01:06:43.991559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 392/781] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:43 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1672 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:44.519690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:44.520060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:44.526788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:44.527247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:44.528019 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:44.528458 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:44.657192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:44.665521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:44.667106 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:44.668310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-166791df-f654-41ca-880a-5575172e0592 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1720 microversion: 2.20 time: 0.152769 Jul 24 01:06:44.668815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 390/782] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:44 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1720 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:45.007986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:45.008555 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:45.015065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.015528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:45.016299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.016737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:45.142229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:45.150793 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3f28b45c-3662-49cd-a3a7-0abe0f6fdb09 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1672 microversion: 2.1 time: 0.146946 Jul 24 01:06:45.151382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 393/783] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:45 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1672 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:45.447775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-19707000-72b1-4d75-b805-9f683c8e3263 req-9197d649-35e2-4ec7-a407-d112e7f5d215 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "name": "network-vif-unplugged", "status": "completed", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:45.455763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-19707000-72b1-4d75-b805-9f683c8e3263 req-9197d649-35e2-4ec7-a407-d112e7f5d215 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.456269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-19707000-72b1-4d75-b805-9f683c8e3263 req-9197d649-35e2-4ec7-a407-d112e7f5d215 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:45.468294 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-19707000-72b1-4d75-b805-9f683c8e3263 req-9197d649-35e2-4ec7-a407-d112e7f5d215 service nova] Creating event network-vif-unplugged:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:06:45.472907 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-19707000-72b1-4d75-b805-9f683c8e3263 req-9197d649-35e2-4ec7-a407-d112e7f5d215 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.028847 Jul 24 01:06:45.474333 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 391/784] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:45 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:45.685934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:45.686536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:45.692809 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.693533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:45.694306 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.694759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:45.827658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:45.836619 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:45.838804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:45.840336 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e02d59f1-df8a-431e-8716-097ebbb69e9c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1723 microversion: 2.20 time: 0.158135 Jul 24 01:06:45.856662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 394/785] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:45 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:45.867716 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "807c7e3d-0b06-42a7-92ae-a82af725e3f4"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:45.868870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:45.875223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.875671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:45.876438 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:45.876866 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.169894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:46.171085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:46.182338 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.183804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.184773 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.185953 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.186815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f5edc79e-47d7-4eab-ab4f-1ad2d2a4dc00 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.329524 Jul 24 01:06:46.187507 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 392/786] 10.222.0.18 () {62 vars in 1363 bytes} [Sun Jul 24 01:06:45 2022] POST /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-volume_attachments => generated 188 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 24 01:06:46.319380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:46.320136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:46.326237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.326686 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.327674 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.328257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.353111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:46.363700 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fffa55db-d91c-4c39-82ce-d76deaab9662 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1672 microversion: 2.1 time: 0.197105 Jul 24 01:06:46.365555 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 395/787] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:46 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1672 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:46.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=5,nova_cell1:UPDATE=5 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:46.748713 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ce814164-9b9b-4c47-88ab-4c8a931e5178 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-volume_attachments/807c7e3d-0b06-42a7-92ae-a82af725e3f4" status: 202 len: 0 microversion: 2.20 time: 0.434913 Jul 24 01:06:46.749295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 393/788] 10.222.0.18 () {60 vars in 1456 bytes} [Sun Jul 24 01:06:46 2022] DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-volume_attachments/807c7e3d-0b06-42a7-92ae-a82af725e3f4 => generated 0 bytes in 436 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 24 01:06:46.807690 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:46.808507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:46.814889 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.815412 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.816242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.816771 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.898990 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f42a6614-a9e7-4a13-bd9f-031ab791eaf4 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "POST /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/action" status: 202 len: 0 microversion: 2.20 time: 0.095679 Jul 24 01:06:46.899509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 396/789] 10.222.0.18 () {62 vars in 1318 bytes} [Sun Jul 24 01:06:46 2022] POST /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/action => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:06:46.907482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:46.908237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:46.915611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.916229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:46.917190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:46.917623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:47.060209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:47.070029 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:47.071993 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:47.073164 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cbf1bd42-1628-4179-863b-a2d9dac558af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1671 microversion: 2.20 time: 0.169549 Jul 24 01:06:47.073713 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 394/790] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:46 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1671 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:47.289758 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-6416eba8-6baa-48b2-a3cf-98a58ff421a6 req-89e61d6c-6ba1-44a1-8dcd-0d83170e96e1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c4a54e8-8220-4b56-bbb7-51402654c743", "name": "network-vif-deleted", "tag": "ffc72e66-3327-493d-a572-31822359f754"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:47.304885 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-6416eba8-6baa-48b2-a3cf-98a58ff421a6 req-89e61d6c-6ba1-44a1-8dcd-0d83170e96e1 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:47.305791 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-6416eba8-6baa-48b2-a3cf-98a58ff421a6 req-89e61d6c-6ba1-44a1-8dcd-0d83170e96e1 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:47.335476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-6416eba8-6baa-48b2-a3cf-98a58ff421a6 req-89e61d6c-6ba1-44a1-8dcd-0d83170e96e1 service nova] Creating event network-vif-deleted:ffc72e66-3327-493d-a572-31822359f754 for instance 1c4a54e8-8220-4b56-bbb7-51402654c743 on hv-cinder-maste Jul 24 01:06:47.341657 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-6416eba8-6baa-48b2-a3cf-98a58ff421a6 req-89e61d6c-6ba1-44a1-8dcd-0d83170e96e1 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064074 Jul 24 01:06:47.342631 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 397/791] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:06:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:06:47.387004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:47.387486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:47.397095 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:47.399556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:47.400710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:47.403975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:47.461403 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:47.461865 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:47.462333 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lazy-loading 'fault' on Instance uuid 1c4a54e8-8220-4b56-bbb7-51402654c743 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:06:47.526811 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a59f5944a0db4710a43a6447019864d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '644464d3831e468f8ab9957bb33817b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:47.534424 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c2496039-c19b-4874-8b8b-c53c7d4c4178 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 200 len: 1326 microversion: 2.1 time: 0.156842 Jul 24 01:06:47.536256 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 395/792] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:47 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 1326 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:06:48.091813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:48.092505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:48.099762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:48.100423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:48.101205 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:48.101637 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:48.238199 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:48.240745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:06:48.250474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:48.252533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:48.253511 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-180d6821-3ba1-4c91-9f7c-a6854290a62b tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.165951 Jul 24 01:06:48.254074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 398/793] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:48 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:48.551529 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:48.552882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:48.559616 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:48.560074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:48.560846 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:48.561289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:48.576070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] HTTP exception thrown: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Request method failure captured: Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 HTTP/1.1 Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm8nnZce8yp0GRDlYqUl0iNdKlyJb5uKKi3sf1HvzqIkN_s6psHRHdLLUWihPvdD8IoiN0VH_RTioVAkBC3TTe8ckleScL7HRvbfiaqAxJGBSZqKAG7x_HZIcGDyreen0LNHvSjWCRPLgXWzIa1ypoyelWd3YaaB39JfrVeWNJQJuw Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:06:48.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_644464d3831e468f8ab9957bb33817b0", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: a59f5944a0db4710a43a6447019864d2 Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:48.579263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '1c4a54e8-8220-4b56-bbb7-51402654c743'} Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:48.580191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:48.581695 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:48.584236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Returning 404 to user: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:48.584236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3aeafcfb-c719-4ca2-8c23-6ce44e28d242 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 404 len: 111 microversion: 2.1 time: 0.031605 Jul 24 01:06:48.584236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 396/794] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:48 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:06:48.589790 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-94858c17-ff81-4521-abe6-d0a4fc8fc0b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '.version_select at 0x7f5e34d511f0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:48.614875 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-94858c17-ff81-4521-abe6-d0a4fc8fc0b0 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-766938946" status: 202 len: 0 microversion: 2.1 time: 0.028935 Jul 24 01:06:48.615483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 399/795] 10.222.0.18 () {58 vars in 1217 bytes} [Sun Jul 24 01:06:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-766938946 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:06:49.285585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:49.287398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:49.304162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:49.306207 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:49.307954 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:49.308901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:49.503653 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:49.511818 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:49.514060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:49.524830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fb537fe1-2b9b-45d6-ac15-8d052c4ceb47 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.244311 Jul 24 01:06:49.525747 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 397/796] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:49 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:50.544664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:50.545360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:50.563744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:50.564412 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:50.565562 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:50.566160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:50.810089 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:50.825643 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:50.827345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:50.828760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-572a8aa6-9644-4ee0-9689-dda0ecaa9906 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.289400 Jul 24 01:06:50.829532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 400/797] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:50 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:51.725017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:51.725531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:51.732336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:51.732794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:51.733579 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:51.734023 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:51.747727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] HTTP exception thrown: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Request method failure captured: Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 HTTP/1.1 Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm8nnZce8yp0GRDlYqUl0iNdKlyJb5uKKi3sf1HvzqIkN_s6psHRHdLLUWihPvdD8IoiN0VH_RTioVAkBC3TTe8ckleScL7HRvbfiaqAxJGBSZqKAG7x_HZIcGDyreen0LNHvSjWCRPLgXWzIa1ypoyelWd3YaaB39JfrVeWNJQJuw Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:06:51.749264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_644464d3831e468f8ab9957bb33817b0", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: a59f5944a0db4710a43a6447019864d2 Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.750737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '1c4a54e8-8220-4b56-bbb7-51402654c743'} Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:51.751690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.753103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.755251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Returning 404 to user: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:51.755251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4bb55d20-d616-42b9-8179-2b92c6b423ea tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 404 len: 111 microversion: 2.1 time: 0.029614 Jul 24 01:06:51.755251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 398/798] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:06:51 2022] DELETE /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:06:51.760399 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:51.761004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 1c4a54e8-8220-4b56-bbb7-51402654c743] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:51.767604 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:51.768098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:51.768804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:51.769231 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:51.784295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] HTTP exception thrown: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Request method failure captured: Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 HTTP/1.1 Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm8nnZce8yp0GRDlYqUl0iNdKlyJb5uKKi3sf1HvzqIkN_s6psHRHdLLUWihPvdD8IoiN0VH_RTioVAkBC3TTe8ckleScL7HRvbfiaqAxJGBSZqKAG7x_HZIcGDyreen0LNHvSjWCRPLgXWzIa1ypoyelWd3YaaB39JfrVeWNJQJuw Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:06:51.786069 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_644464d3831e468f8ab9957bb33817b0", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: a59f5944a0db4710a43a6447019864d2 Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.787733 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '1c4a54e8-8220-4b56-bbb7-51402654c743'} Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:51.788626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. Jul 24 01:06:51.790854 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:51.792250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Returning 404 to user: Instance 1c4a54e8-8220-4b56-bbb7-51402654c743 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:51.792250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8b5c9ecf-8d6a-4f4f-af1f-5d205654c90b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743" status: 404 len: 111 microversion: 2.1 time: 0.030962 Jul 24 01:06:51.792250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 401/799] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:51 2022] GET /compute/v2.1/servers/1c4a54e8-8220-4b56-bbb7-51402654c743 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:06:51.848191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:51.849415 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:51.863076 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:51.863531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:51.865358 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:51.865891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:52.056998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:52.065472 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:52.067634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:52.069487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cf16384d-2d72-4190-9159-ff546ae22230 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.225843 Jul 24 01:06:52.070036 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 399/800] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:51 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:52.976245 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:52.976770 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:52.983957 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:52.984677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:52.985662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:52.986313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:53.001576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] HTTP exception thrown: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Request method failure captured: Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa HTTP/1.1 Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm8nnZce8yp0GRDlYqUl0iNdKlyJb5uKKi3sf1HvzqIkN_s6psHRHdLLUWihPvdD8IoiN0VH_RTioVAkBC3TTe8ckleScL7HRvbfiaqAxJGBSZqKAG7x_HZIcGDyreen0LNHvSjWCRPLgXWzIa1ypoyelWd3YaaB39JfrVeWNJQJuw Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:06:53.003253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_644464d3831e468f8ab9957bb33817b0", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: a59f5944a0db4710a43a6447019864d2 Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.005503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '64ba3b57-16f8-4b6f-b0a6-d0141afc0faa'} Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:53.006484 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.008123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.009476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Returning 404 to user: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:53.009476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ccaa4f4f-b297-45b7-9dc8-813bedf1edf5 tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 404 len: 111 microversion: 2.1 time: 0.034475 Jul 24 01:06:53.009476 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 402/801] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:06:52 2022] DELETE /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:06:53.016205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:53.016820 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] [instance: 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:53.024269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:53.024723 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:53.025491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:53.025931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:53.041262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] HTTP exception thrown: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Request method failure captured: Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa HTTP/1.1 Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm8nnZce8yp0GRDlYqUl0iNdKlyJb5uKKi3sf1HvzqIkN_s6psHRHdLLUWihPvdD8IoiN0VH_RTioVAkBC3TTe8ckleScL7HRvbfiaqAxJGBSZqKAG7x_HZIcGDyreen0LNHvSjWCRPLgXWzIa1ypoyelWd3YaaB39JfrVeWNJQJuw Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:06:53.042862 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/644464d3831e468f8ab9957bb33817b0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_644464d3831e468f8ab9957bb33817b0", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 644464d3831e468f8ab9957bb33817b0 Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeTestJSON-910833463 Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: a59f5944a0db4710a43a6447019864d2 Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeTestJSON-910833463-project Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.044300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '64ba3b57-16f8-4b6f-b0a6-d0141afc0faa'} Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:06:53.045103 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. Jul 24 01:06:53.046556 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:06:53.048873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] Returning 404 to user: Instance 64ba3b57-16f8-4b6f-b0a6-d0141afc0faa could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:06:53.048873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-df15a5e3-03da-4b24-baa8-0447396a3a2b tempest-AttachVolumeTestJSON-910833463 tempest-AttachVolumeTestJSON-910833463-project] 10.222.0.18 "GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa" status: 404 len: 111 microversion: 2.1 time: 0.031973 Jul 24 01:06:53.048873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 400/802] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:06:53 2022] GET /compute/v2.1/servers/64ba3b57-16f8-4b6f-b0a6-d0141afc0faa => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:06:53.088253 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:53.088855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:53.095078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:53.095531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:53.096292 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:53.096730 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:53.245714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:53.254950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:53.256457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:53.257668 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e0d48cfc-4cdd-47bb-a040-70d7a7650620 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.173353 Jul 24 01:06:53.258226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 403/803] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:53 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:54.288045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:54.293933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:54.313201 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:54.313860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:54.314891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:54.315477 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:54.553717 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:54.563219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:54.567850 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:54.575978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-23ff8c7f-47f7-4f5e-a7ad-3e1ab49e61f6 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.301153 Jul 24 01:06:54.576820 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 401/804] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:54 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:55.594287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:55.594929 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:55.605548 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:55.606049 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:55.618806 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:55.619352 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:55.967194 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:55.984781 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:55.986285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:55.987548 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f6cccdb3-d24b-425b-aec9-20b182b502ba tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.397098 Jul 24 01:06:55.993467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 404/805] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:55 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:57.010180 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:57.011844 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:57.065212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:57.065659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:57.066724 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:57.067521 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:57.306515 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:57.332395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:57.335823 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:57.337595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3cf786a9-9751-4dcb-aa7e-0138e8900e44 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.331837 Jul 24 01:06:57.338906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 402/806] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:57 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:06:58.357708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:58.358537 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:58.366279 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:58.366736 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:58.367549 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:58.367986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:58.528436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:58.537356 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:58.539401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:58.540964 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-77d7b8f0-8dee-4823-a615-bcc0c4c77cf1 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.188229 Jul 24 01:06:58.541544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 405/807] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:58 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:06:59.248273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3d9703ea-b97d-45a1-af1d-1093657afc2f tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Action: 'create', calling method: .version_select at 0x7f5e3035e670>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-144655519", "metadata": {"Type": "work"}}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:06:59.565360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:59.565980 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:06:59.574033 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:59.574521 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:59.575372 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:06:59.576597 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:06:59.610793 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3d9703ea-b97d-45a1-af1d-1093657afc2f tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.480381 Jul 24 01:06:59.611719 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 403/808] 10.222.0.18 () {60 vars in 1155 bytes} [Sun Jul 24 01:06:59 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 482 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 24 01:06:59.626202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c02751e9-10e7-40da-aaf3-9f68c0b428fa tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Calling method '.version_select at 0x7f5e3035e940>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:06:59.683538 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c02751e9-10e7-40da-aaf3-9f68c0b428fa tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183" status: 200 len: 352 microversion: 2.1 time: 0.064524 Jul 24 01:06:59.684105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 404/809] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:06:59 2022] GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 => generated 352 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:06:59.732061 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:59.741124 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:59.742620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:06:59.743837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-59a6ddb0-c0a2-429f-bdff-aa62922273ff tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.183037 Jul 24 01:06:59.744393 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 406/810] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:06:59 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:00.697329 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3188f39f-0bbe-45ec-81e5-074dcc025771 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Calling method '.version_select at 0x7f5e3023ec10>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:00.744859 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3188f39f-0bbe-45ec-81e5-074dcc025771 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183" status: 200 len: 353 microversion: 2.1 time: 0.052104 Jul 24 01:07:00.745426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 405/811] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:00 2022] GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 => generated 353 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:00.756816 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5f040207-2e2a-4c4c-a99d-167f8a38f0dc tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Calling method '.version_select at 0x7f5e3046fe50>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:00.761054 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:00.761683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:00.768987 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:00.769590 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:00.771423 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:00.771889 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:00.805976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5f040207-2e2a-4c4c-a99d-167f8a38f0dc tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183" status: 200 len: 353 microversion: 2.1 time: 0.053485 Jul 24 01:07:00.806547 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 407/812] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:00 2022] GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 => generated 353 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:00.824180 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-99c081a7-92f7-4028-aebc-81aa3f266985 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Calling method '.version_select at 0x7f5e304a5790>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:00.887906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-99c081a7-92f7-4028-aebc-81aa3f266985 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "DELETE /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183" status: 202 len: 0 microversion: 2.1 time: 0.067480 Jul 24 01:07:00.888561 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 408/813] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:07:00 2022] DELETE /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 => generated 0 bytes in 69 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:07:00.896339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6fe60b16-ca4d-48f9-a65d-39a9e5f0c89e tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Calling method '.version_select at 0x7f5e34d724c0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:00.908741 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:00.917070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:00.918572 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:00.919791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-37037915-80a7-4664-8abf-8d45bac99efd tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.162459 Jul 24 01:07:00.920350 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 406/814] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:00 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:07:00.945042 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6fe60b16-ca4d-48f9-a65d-39a9e5f0c89e tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183" status: 200 len: 352 microversion: 2.1 time: 0.052440 Jul 24 01:07:00.945676 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 409/815] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:00 2022] GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 => generated 352 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:07:01.938759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:01.939387 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:01.946172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:01.947031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:01.947806 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:01.948250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:01.957734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c3967c9b-b376-42a0-a47b-e6c9f7faa8b0 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Calling method '.version_select at 0x7f5e3046f820>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:01.986915 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-c3967c9b-b376-42a0-a47b-e6c9f7faa8b0 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] HTTP exception thrown: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c3967c9b-b376-42a0-a47b-e6c9f7faa8b0 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Request method failure captured: Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 HTTP/1.1 Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jszn51Hdxa0ORZEfD0vJnDabfEZIWpHGkoI7QhsEARfG6ftYtNzbqKHt3gHlyWLzpFiJFhkWrnS8kiDRWhz40zUkTO__IMSiBmSLeVxIoD7qf02zjyU94FKk8KcOegphxpkYxrjrGEsmtnTi3sVtJa-kDhXlyEdZ4q_1xMx60Bullc Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 74c0c8908908487bbf78fc18aca63c9b Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-VolumesGetTestJSON-812080621 Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:01.988918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/74c0c8908908487bbf78fc18aca63c9b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/74c0c8908908487bbf78fc18aca63c9b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/74c0c8908908487bbf78fc18aca63c9b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_74c0c8908908487bbf78fc18aca63c9b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-VolumesGetTestJSON-812080621 Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 74c0c8908908487bbf78fc18aca63c9b Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-VolumesGetTestJSON-812080621 Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-VolumesGetTestJSON-812080621-project Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: a618f13fb1554a92b3c0a03eecf40a93 Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-VolumesGetTestJSON-812080621-project Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '0438eb67-38b9-4012-b91b-bf5f6cbf6183'} Jul 24 01:07:01.991176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. (HTTP 404) (Request-ID: req-0da4da77-5d41-40f9-89e7-176b13d94462) Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:01.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 24 01:07:01.993188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 24 01:07:01.994294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c3967c9b-b376-42a0-a47b-e6c9f7faa8b0 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] Returning 404 to user: Volume 0438eb67-38b9-4012-b91b-bf5f6cbf6183 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c3967c9b-b376-42a0-a47b-e6c9f7faa8b0 tempest-VolumesGetTestJSON-812080621 tempest-VolumesGetTestJSON-812080621-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183" status: 404 len: 109 microversion: 2.1 time: 0.037502 Jul 24 01:07:01.995493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 410/816] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:01 2022] GET /compute/v2.1/os-volumes/0438eb67-38b9-4012-b91b-bf5f6cbf6183 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 24 01:07:01.996814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:01.997045 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:02.003944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:02.004406 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:02.005817 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:02.006265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:02.034273 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-614bdb59-b258-4530-b969-01c945627a98 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/ips" status: 200 len: 149 microversion: 2.1 time: 0.040896 Jul 24 01:07:02.034515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 411/817] 10.222.0.18 () {58 vars in 1247 bytes} [Sun Jul 24 01:07:01 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/ips => generated 149 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:02.102433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:02.110449 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:02.112015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:02.113208 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-dbf45969-2f3b-4d41-aaae-bbad832587af tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.178354 Jul 24 01:07:02.113764 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 407/818] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:01 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:03.132613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:03.133330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:03.140226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:03.140684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:03.141505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:03.141898 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:03.293765 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:03.307012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:03.308529 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:03.309694 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-75fbf940-8672-46cf-82c8-94106a02efd2 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.181216 Jul 24 01:07:03.310247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 412/819] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:03 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:03.401694 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "00f25028-f8e1-414b-aba1-e8159962b403"}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:03.403885 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:03.414528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:03.415054 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:03.415839 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:03.416455 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:03.694995 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-396a4bd2-8f28-4df5-89b6-8d46b5c1fdc6 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "POST /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.297267 Jul 24 01:07:03.696011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 408/820] 10.222.0.18 () {60 vars in 1322 bytes} [Sun Jul 24 01:07:03 2022] POST /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments => generated 194 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:04.329215 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:04.329848 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:04.336944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:04.337403 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:04.338240 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:04.338625 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:04.488719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:04.497647 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:04.499208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:04.500702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9c98f336-4885-4db4-accf-0d5cf6f06fb0 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.175627 Jul 24 01:07:04.501615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 413/821] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:04 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:04.680718 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-c249a1c3-160f-4e2d-b670-106d36e491cd req-23d0474d-e20a-40bb-829f-7122cacaab09 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "name": "network-vif-plugged", "status": "completed", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:04.689881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-c249a1c3-160f-4e2d-b670-106d36e491cd req-23d0474d-e20a-40bb-829f-7122cacaab09 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:04.690459 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-c249a1c3-160f-4e2d-b670-106d36e491cd req-23d0474d-e20a-40bb-829f-7122cacaab09 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:04.702512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-c249a1c3-160f-4e2d-b670-106d36e491cd req-23d0474d-e20a-40bb-829f-7122cacaab09 service nova] Creating event network-vif-plugged:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:07:04.707344 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-c249a1c3-160f-4e2d-b670-106d36e491cd req-23d0474d-e20a-40bb-829f-7122cacaab09 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031146 Jul 24 01:07:04.707927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 409/822] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:05.519402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:05.519935 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:05.527280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:05.527734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:05.528505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:05.528939 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:05.685717 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:05.694828 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:05.696347 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:05.697694 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b3ed5abf-b069-4488-bbfe-0e77a1227705 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.182400 Jul 24 01:07:05.698301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 414/823] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:05 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:06.717670 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:06.718469 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:06.729482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:06.729482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:06.729482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:06.729482 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:06.885681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:06.896164 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:06.898268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:06.899718 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e8c4c133-f425-42bc-ba69-0eb31c5ef01c tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1729 microversion: 2.20 time: 0.187157 Jul 24 01:07:06.912860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 410/824] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:06 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1729 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:07.919216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:07.919819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:07.926310 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:07.926752 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:07.927658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:07.928085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:08.066356 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:08.074529 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:08.076087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:08.077259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-26346fa4-62e4-4c09-b03f-3e9eca392e2e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1716 microversion: 2.20 time: 0.161971 Jul 24 01:07:08.078023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 415/825] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:07 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1716 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:11.342537 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:11.343120 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:11.345504 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:11.352884 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:11.353374 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:11.354153 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:11.354594 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:11.452089 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lazy-loading 'services' on Instance uuid e985f6e0-7865-4447-868d-568639afe87b {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:07:11.518661 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:INSERT=4,nova_cell1:SELECT=90,nova_cell1:UPDATE=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:11.530015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-be52e637-191f-4505-9212-3fd820fac908 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments/00f25028-f8e1-414b-aba1-e8159962b403" status: 202 len: 0 microversion: 2.1 time: 0.191263 Jul 24 01:07:11.530610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 411/826] 10.222.0.18 () {58 vars in 1415 bytes} [Sun Jul 24 01:07:11 2022] DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments/00f25028-f8e1-414b-aba1-e8159962b403 => generated 0 bytes in 193 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:07:11.539158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:11.540189 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:11.542544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=51,nova_api:DELETE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:11.547910 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:11.548358 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:11.549141 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:11.549617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:11.581134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8d307719-41b7-4350-a14c-7e31824867bf tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046029 Jul 24 01:07:11.581699 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 416/827] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:07:11 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:12.594375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:12.595413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:12.602841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:12.603298 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:12.604075 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:12.604515 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:12.634369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f655fbb1-8d12-48ee-b15e-b8fe171d2e9e tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045614 Jul 24 01:07:12.634964 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 412/828] 10.222.0.18 () {58 vars in 1301 bytes} [Sun Jul 24 01:07:12 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 24 01:07:12.695545 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:12.696114 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:12.704182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:12.704732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:12.705500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:12.706120 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:12.731989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:07:12.825850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8817e08f-ce7d-4323-8509-082bfdf2741c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 204 len: 0 microversion: 2.1 time: 0.135641 Jul 24 01:07:12.830252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 417/829] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:07:12 2022] DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:07:12.837453 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:12.838351 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:12.845628 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:12.846566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:12.847749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:12.848473 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:12.992918 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:13.000375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-daab2ad0-8602-45ab-84ee-1f6d9abcb2d8 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1680 microversion: 2.1 time: 0.168977 Jul 24 01:07:13.000930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 413/830] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:12 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1680 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:07:14.017784 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:14.018477 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:14.025055 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:14.025508 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:14.026277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:14.026713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:14.167152 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:14.175678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0bbf84bc-5627-4c20-92f3-cc681b784b9f tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1680 microversion: 2.1 time: 0.161964 Jul 24 01:07:14.186152 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 418/831] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:14 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1680 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:15.194239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:15.195481 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:15.202375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:15.202974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:15.203775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:15.204248 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:15.267679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:15.276633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a4455225-1f9b-4b58-b937-8156aed87566 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1613 microversion: 2.1 time: 0.086779 Jul 24 01:07:15.277214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 414/832] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:15 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1613 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:07:15.456965 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-ffb51aed-4b24-4f88-b141-a8452239c602 req-848381a7-fe42-4276-b4ec-9bb0ce82fc12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e985f6e0-7865-4447-868d-568639afe87b", "name": "network-vif-deleted", "tag": "87ad2158-30f6-4543-a57f-4f732fe5c551"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:15.466483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ffb51aed-4b24-4f88-b141-a8452239c602 req-848381a7-fe42-4276-b4ec-9bb0ce82fc12 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:15.467026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ffb51aed-4b24-4f88-b141-a8452239c602 req-848381a7-fe42-4276-b4ec-9bb0ce82fc12 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:15.479985 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-ffb51aed-4b24-4f88-b141-a8452239c602 req-848381a7-fe42-4276-b4ec-9bb0ce82fc12 service nova] Creating event network-vif-deleted:87ad2158-30f6-4543-a57f-4f732fe5c551 for instance e985f6e0-7865-4447-868d-568639afe87b on hv-cinder-maste Jul 24 01:07:15.484884 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-ffb51aed-4b24-4f88-b141-a8452239c602 req-848381a7-fe42-4276-b4ec-9bb0ce82fc12 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033223 Jul 24 01:07:15.485900 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 419/833] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:16.293565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:16.294183 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:16.300990 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:16.301593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:16.302689 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:16.303166 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:16.370795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:16.378360 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f2b7dd51-0e99-4ef3-89f7-d3ca20a4ed42 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1613 microversion: 2.1 time: 0.088722 Jul 24 01:07:16.378961 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 415/834] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:16 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1613 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:17.396744 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:17.397448 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:17.403835 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:17.404285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:17.405115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:17.405557 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:17.466971 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f471531dac249d994f864102f7b9d3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c68e39bc1724fda980be3463bbf7d7b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:17.474588 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-59fd8810-b633-400c-86ea-7c82e5e13272 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 200 len: 1326 microversion: 2.1 time: 0.082537 Jul 24 01:07:17.475185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 420/835] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:17 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 1326 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:18.491339 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:18.491872 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:18.498435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:18.498922 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:18.499693 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:18.500130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:18.513643 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b HTTP/1.1 Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:18.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:18.516840 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'e985f6e0-7865-4447-868d-568639afe87b'} Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:18.517749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:18.519354 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:18.520662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:18.520662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d8cb0bf5-2b6b-4d09-8e9a-a8279b1c69bb tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 404 len: 111 microversion: 2.1 time: 0.029413 Jul 24 01:07:18.520662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 416/836] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:18 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:18.526480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-dce15c94-99a2-4070-9674-e9ab057bd027 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '.version_select at 0x7f5e34d724c0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:18.543486 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-dce15c94-99a2-4070-9674-e9ab057bd027 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1567354850" status: 202 len: 0 microversion: 2.1 time: 0.020893 Jul 24 01:07:18.544038 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 421/837] 10.222.0.18 () {58 vars in 1220 bytes} [Sun Jul 24 01:07:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1567354850 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:07:20.286680 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:20.287287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:20.293976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:20.294422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:20.295226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:20.295662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:20.309714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b HTTP/1.1 Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:20.311357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.313026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'e985f6e0-7865-4447-868d-568639afe87b'} Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:20.313831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.315016 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.316367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:20.316367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-026a68b9-4cfe-4277-806a-ed36a4a85d09 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 404 len: 111 microversion: 2.1 time: 0.029987 Jul 24 01:07:20.316367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 417/838] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:07:20 2022] DELETE /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:20.322467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:20.323113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: e985f6e0-7865-4447-868d-568639afe87b] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:20.330406 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:20.331016 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:20.331786 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:20.332223 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:20.337015 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:20.346584 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b HTTP/1.1 Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:20.348415 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.349774 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'e985f6e0-7865-4447-868d-568639afe87b'} Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:20.350525 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. Jul 24 01:07:20.351777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:20.352949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance e985f6e0-7865-4447-868d-568639afe87b could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:20.352949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5c7272c4-bc5f-420c-9ebf-e77019b1826c tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b" status: 404 len: 111 microversion: 2.1 time: 0.032280 Jul 24 01:07:20.352949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 422/839] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:20 2022] GET /compute/v2.1/servers/e985f6e0-7865-4447-868d-568639afe87b => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:21.504417 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:21.504901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:21.511568 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.512023 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.512796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.513235 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.528573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 HTTP/1.1 Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:21.530112 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.531564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '4693fbdb-0311-4f58-a0b7-e338e3369224'} Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:21.532456 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.533898 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.535546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:21.535546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b8677a13-641e-4e3f-a19b-593511ff82e3 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 404 len: 111 microversion: 2.1 time: 0.031012 Jul 24 01:07:21.535546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 418/840] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:07:21 2022] DELETE /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:21.541755 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:21.542394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 4693fbdb-0311-4f58-a0b7-e338e3369224] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:21.553709 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.554163 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.554961 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.555392 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.572339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 HTTP/1.1 Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:21.573825 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.575341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '4693fbdb-0311-4f58-a0b7-e338e3369224'} Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:21.576139 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. Jul 24 01:07:21.578913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.581418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance 4693fbdb-0311-4f58-a0b7-e338e3369224 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:21.581418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9ba5b022-0425-4e05-b9ca-846612c36965 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224" status: 404 len: 111 microversion: 2.1 time: 0.037745 Jul 24 01:07:21.581418 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 423/841] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:21 2022] GET /compute/v2.1/servers/4693fbdb-0311-4f58-a0b7-e338e3369224 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:21.585017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:21.585512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:21.593476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.594206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.595023 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.595466 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.610696 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 HTTP/1.1 Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:21.612273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.613888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'fd889446-7a8d-4e52-91a5-22b75a923a21'} Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:21.615499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.618367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:21.618367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6dbf9265-8824-4388-94ff-e9a638c1a433 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 404 len: 111 microversion: 2.1 time: 0.033240 Jul 24 01:07:21.618367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 419/842] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:07:21 2022] DELETE /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:21.623009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:21.623573 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: fd889446-7a8d-4e52-91a5-22b75a923a21] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:21.629741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.630868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.631636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:21.632067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:21.802986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 HTTP/1.1 Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:21.804520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.806097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'fd889446-7a8d-4e52-91a5-22b75a923a21'} Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:21.807006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. Jul 24 01:07:21.808454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:21.810472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance fd889446-7a8d-4e52-91a5-22b75a923a21 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:21.810472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9439a5d3-2a64-4144-8864-0c911da90023 tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21" status: 404 len: 111 microversion: 2.1 time: 0.186856 Jul 24 01:07:21.810472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 424/843] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:21 2022] GET /compute/v2.1/servers/fd889446-7a8d-4e52-91a5-22b75a923a21 => generated 111 bytes in 188 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:22.955015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:22.955607 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:22.963131 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:22.963597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:22.964649 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:22.965098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:22.979925 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a HTTP/1.1 Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:22.981484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:22.982809 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '1c006a24-d0c5-4ab6-9ce4-259e05a9c56a'} Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:22.983543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:22.984729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:22.985974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:22.985974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-cd2563f9-8f86-4fc5-a7b0-db349aa3d0aa tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 404 len: 111 microversion: 2.1 time: 0.041259 Jul 24 01:07:22.985974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 420/844] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:07:22 2022] DELETE /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:22.991739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:22.992345 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] [instance: 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:22.999236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:22.999765 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:23.000610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:23.001113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:23.015757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] HTTP exception thrown: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Request method failure captured: Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a HTTP/1.1 Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JnLasgkzwb4MbeEfp7j6utFum-qPAXaKnND1xkyBxdsNj6-q2GFNMJXZO_5CpP2LeP_L5EtT0MwQu9a2wr5GyJrgt7xpYJVF8SN1qJPRn_YlLOaFrZUkLDhydsC2B_bGm47ZkfKh3y9SZ0tKVCiRu1oazFKw4HzpimD6-TDCEeX7x8 Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:23.017295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/3c68e39bc1724fda980be3463bbf7d7b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_3c68e39bc1724fda980be3463bbf7d7b", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 3c68e39bc1724fda980be3463bbf7d7b Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1351003122 Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 8f471531dac249d994f864102f7b9d3f Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeNegativeTest-1351003122-project Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:23.018600 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '1c006a24-d0c5-4ab6-9ce4-259e05a9c56a'} Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:23.019375 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. Jul 24 01:07:23.020677 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:23.023750 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] Returning 404 to user: Instance 1c006a24-d0c5-4ab6-9ce4-259e05a9c56a could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:23.024914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e0759bcf-079b-4f62-895c-c723d02eedec tempest-AttachVolumeNegativeTest-1351003122 tempest-AttachVolumeNegativeTest-1351003122-project] 10.222.0.18 "GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a" status: 404 len: 111 microversion: 2.1 time: 0.037308 Jul 24 01:07:23.025475 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 425/845] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:22 2022] GET /compute/v2.1/servers/1c006a24-d0c5-4ab6-9ce4-259e05a9c56a => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:29.524742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4390be6a-bda2-4178-9d93-68898628048b tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e308e3ee0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:29.575212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4390be6a-bda2-4178-9d93-68898628048b tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/0becfd49-2d84-4121-888d-cc98147e4222" status: 200 len: 350 microversion: 2.1 time: 0.055274 Jul 24 01:07:29.575772 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 421/846] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:29 2022] GET /compute/v2.1/os-volumes/0becfd49-2d84-4121-888d-cc98147e4222 => generated 350 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:30.912875 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3f696b05-b8ed-4260-8562-6f5a99a8f888 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e34d550d0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:30.959732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3f696b05-b8ed-4260-8562-6f5a99a8f888 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/3b4c1117-e3c7-4c53-a044-ef641cebeabc" status: 200 len: 351 microversion: 2.1 time: 0.050590 Jul 24 01:07:30.960274 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 426/847] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:30 2022] GET /compute/v2.1/os-volumes/3b4c1117-e3c7-4c53-a044-ef641cebeabc => generated 351 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:31.378522 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-517375826", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "networks": [{"uuid": "c741dacb-eb73-4391-9009-40d0df6ad4b9"}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:31.430433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] validate_networks() for [('c741dacb-eb73-4391-9009-40d0df6ad4b9', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:07:32.072301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:07:32.072728 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:07:32.073201 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:07:32.073610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:07:32.074026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:07:32.083452 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:32.083907 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:32.084707 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:32.085140 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:32.101429 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Getting quotas for project 54b31a874a6b4ec48f9a938d817c4637. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:07:32.105367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Getting quotas for user ac64444a4b794026a5e95584d31f1c04 and project 54b31a874a6b4ec48f9a938d817c4637. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:07:32.116257 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:07:32.118468 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:07:32.163122 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-53a19637-583f-48c6-bef2-0148a8886ea3 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.788828 Jul 24 01:07:32.163764 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 422/848] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:07:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 790 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:07:32.174277 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-239ad88c-f19e-40ee-bac0-66195f15a4e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.174896 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-239ad88c-f19e-40ee-bac0-66195f15a4e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:32.229210 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-239ad88c-f19e-40ee-bac0-66195f15a4e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:32.238360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-239ad88c-f19e-40ee-bac0-66195f15a4e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1213 microversion: 2.1 time: 0.067786 Jul 24 01:07:32.240062 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 427/849] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1213 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:32.276437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-488bdb64-bf41-4e4b-a9fe-d8cdaf50846e tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e3038a5e0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.327371 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-488bdb64-bf41-4e4b-a9fe-d8cdaf50846e tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/eb3953ed-bdd5-4fd3-a6e6-de43e9af1177" status: 200 len: 351 microversion: 2.1 time: 0.054637 Jul 24 01:07:32.327928 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 423/850] 10.222.0.18 () {58 vars in 1244 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes/eb3953ed-bdd5-4fd3-a6e6-de43e9af1177 => generated 351 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:32.340956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4c43e30f-dff7-4df3-a946-b46c01e0d563 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e34d558b0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.430262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4c43e30f-dff7-4df3-a946-b46c01e0d563 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.094189 Jul 24 01:07:32.431020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 428/851] 10.222.0.18 () {58 vars in 1133 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:32.445291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-718dffed-8bce-400b-a9cc-c2241eb81075 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e3033e040>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.529652 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-718dffed-8bce-400b-a9cc-c2241eb81075 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.089888 Jul 24 01:07:32.530240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 424/852] 10.222.0.18 () {58 vars in 1148 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:32.542819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3df00816-a2f1-434a-ad24-832fc7988bb6 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e32989f70>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.626651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3df00816-a2f1-434a-ad24-832fc7988bb6 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.088006 Jul 24 01:07:32.627295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 429/853] 10.222.0.18 () {58 vars in 1133 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:07:32.639316 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-831cfcfb-7f62-4672-bfc2-9eed1d2732aa tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e302bd700>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.720750 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-831cfcfb-7f62-4672-bfc2-9eed1d2732aa tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.085096 Jul 24 01:07:32.721316 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 425/854] 10.222.0.18 () {58 vars in 1166 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:32.733743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-835384f9-7f26-43ea-8fdb-415e758e4535 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e3042c3a0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.819349 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-835384f9-7f26-43ea-8fdb-415e758e4535 tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.089351 Jul 24 01:07:32.819914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 430/855] 10.222.0.18 () {58 vars in 1169 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:32.831502 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5113d1b4-75aa-4cf7-980d-c971285ca3db tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e3029d0d0>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:32.909856 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5113d1b4-75aa-4cf7-980d-c971285ca3db tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.081976 Jul 24 01:07:32.910420 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 426/856] 10.222.0.18 () {58 vars in 1154 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:32.922022 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-29f2452a-87d4-4693-a5f5-022bfac4b21c tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e34d901f0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:33.004980 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-29f2452a-87d4-4693-a5f5-022bfac4b21c tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.086587 Jul 24 01:07:33.005546 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 431/857] 10.222.0.18 () {58 vars in 1187 bytes} [Sun Jul 24 01:07:32 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:33.016857 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-011b56ad-45b4-42b2-914c-9028f347ed0c tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] Calling method '.version_select at 0x7f5e3023e310>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:33.100975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-011b56ad-45b4-42b2-914c-9028f347ed0c tempest-VolumesTestJSON-1781787590 tempest-VolumesTestJSON-1781787590-project] 10.222.0.18 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.087817 Jul 24 01:07:33.101547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 427/858] 10.222.0.18 () {58 vars in 1154 bytes} [Sun Jul 24 01:07:33 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:33.257905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:33.258497 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:33.260549 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=7 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:33.276324 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:33.276775 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:33.277546 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:33.277979 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:33.342856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:33.351760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e1cdf9f2-bac9-408f-a6ae-07ab1ad1b018 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1283 microversion: 2.1 time: 0.098093 Jul 24 01:07:33.352633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 432/859] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:33 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1283 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:34.370178 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:34.370868 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:34.377554 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:34.378150 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:34.378980 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:34.379467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:34.488125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-55bca0e4-8be7-407d-8cdc-6f5ef414ecfe req-94fe2d1b-8846-4c0b-9c40-70bc932237bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d56f41a6-596a-4045-9f75-d1972a1479f8", "tag": "73b83928-1357-4598-a9a2-723487673e12"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:34.498940 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-55bca0e4-8be7-407d-8cdc-6f5ef414ecfe req-94fe2d1b-8846-4c0b-9c40-70bc932237bc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:34.499732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-55bca0e4-8be7-407d-8cdc-6f5ef414ecfe req-94fe2d1b-8846-4c0b-9c40-70bc932237bc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:34.513436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-55bca0e4-8be7-407d-8cdc-6f5ef414ecfe req-94fe2d1b-8846-4c0b-9c40-70bc932237bc service nova] Creating event network-changed:73b83928-1357-4598-a9a2-723487673e12 for instance d56f41a6-596a-4045-9f75-d1972a1479f8 on hv-cinder-maste Jul 24 01:07:34.518678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-55bca0e4-8be7-407d-8cdc-6f5ef414ecfe req-94fe2d1b-8846-4c0b-9c40-70bc932237bc service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034305 Jul 24 01:07:34.519482 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 433/860] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:07:34.537679 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:34.545623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f65bf6e2-73db-41a2-b287-b4902f30bf90 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.179832 Jul 24 01:07:34.546199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 428/861] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:34 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:35.563959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:35.565270 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:35.572913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:35.573367 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:35.574144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:35.574585 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:35.708837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:35.717448 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-77e25a20-81f7-4463-b1ca-3b3325218cdd tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.158211 Jul 24 01:07:35.718106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 434/862] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:35 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:36.737414 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:36.738113 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:36.745175 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:36.745712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:36.746547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:36.747017 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:36.881523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:36.889343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-50f4ac6b-3904-4a3c-bcb3-e8b0a2da9b82 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.155808 Jul 24 01:07:36.890147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 429/863] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:36 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:37.908163 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:37.909557 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:37.919594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:37.920324 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:37.921170 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:37.921994 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:38.077528 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:38.088300 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-47dd58c5-9475-4a03-bf82-03f0470197b5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.184558 Jul 24 01:07:38.089205 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 435/864] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:37 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:39.105366 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:39.108772 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:39.116324 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:39.116926 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:39.117772 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:39.118387 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:39.255266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:39.263674 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7cb2d37c-097d-46a1-aa55-637c3ce7616a tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.162291 Jul 24 01:07:39.264250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 430/865] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:39 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:07:40.027702 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-0894eecb-d1ef-4824-b10c-600025e4f2b7 req-8d65b12e-0b61-499b-b6a4-95e0bad97cc0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d56f41a6-596a-4045-9f75-d1972a1479f8", "name": "network-vif-plugged", "status": "completed", "tag": "73b83928-1357-4598-a9a2-723487673e12"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:40.037099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-0894eecb-d1ef-4824-b10c-600025e4f2b7 req-8d65b12e-0b61-499b-b6a4-95e0bad97cc0 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.037796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-0894eecb-d1ef-4824-b10c-600025e4f2b7 req-8d65b12e-0b61-499b-b6a4-95e0bad97cc0 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:40.049944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-0894eecb-d1ef-4824-b10c-600025e4f2b7 req-8d65b12e-0b61-499b-b6a4-95e0bad97cc0 service nova] Creating event network-vif-plugged:73b83928-1357-4598-a9a2-723487673e12 for instance d56f41a6-596a-4045-9f75-d1972a1479f8 on hv-cinder-maste Jul 24 01:07:40.055790 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-0894eecb-d1ef-4824-b10c-600025e4f2b7 req-8d65b12e-0b61-499b-b6a4-95e0bad97cc0 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032002 Jul 24 01:07:40.056381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 436/866] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:40.285437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:40.288415 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:40.300069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.300069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:40.300069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.300069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:40.433307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:40.440680 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-93c5eff2-5fee-4b81-8981-778c7cf5829d tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.162615 Jul 24 01:07:40.441242 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 431/867] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:40 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:40.890063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:40.891097 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:40.898078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.898531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:40.899328 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.899768 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:40.930244 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-599a7dc7-47af-4e76-84ba-2a439263e9bf tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.044488 Jul 24 01:07:40.931084 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 437/868] 10.222.0.18 () {60 vars in 1342 bytes} [Sun Jul 24 01:07:40 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:40.991002 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:40.991497 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:40.997375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.998004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:40.998778 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:40.999215 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:41.025598 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:07:41.111683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a6ad555d-a6e9-4848-83cb-ce3e2ae52e11 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 204 len: 0 microversion: 2.20 time: 0.124224 Jul 24 01:07:41.112510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 432/869] 10.222.0.18 () {60 vars in 1279 bytes} [Sun Jul 24 01:07:40 2022] DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 24 01:07:41.120732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:41.121341 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:41.128344 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:41.128874 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:41.129715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:41.130261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:41.268376 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:41.278631 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:41.280956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:41.282691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-145c99bc-8e75-415a-9e74-1a3467722235 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.165474 Jul 24 01:07:41.283451 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 438/870] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:41 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:41.457992 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:41.459190 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:41.461599 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:41.466717 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:41.467189 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:41.467942 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:41.468368 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:41.597035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:41.604483 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3d196f02-cee2-4af7-874e-d4aa0423e448 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1313 microversion: 2.1 time: 0.150786 Jul 24 01:07:41.605051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 433/871] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:41 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:42.301030 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:42.301796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:42.305356 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:42.305994 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:42.312226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:42.312670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:42.313430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:42.313855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:42.467502 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:42.476331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:42.477889 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:42.479098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1db23aa4-84ed-40ed-8b9f-310ebfc026b8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.181903 Jul 24 01:07:42.479653 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 439/872] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:42 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:42.622312 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:42.623033 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:42.625098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:42.626071 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:42.632814 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:42.633466 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:42.634250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:42.634689 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:42.639581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:42.779812 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:42.787915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-04b1a833-dee4-48de-9e70-45f03f70794c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1497 microversion: 2.1 time: 0.169522 Jul 24 01:07:42.788488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 434/873] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:42 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:43.426624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:43.427178 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:43.434350 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:43.434814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:43.435620 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:43.436054 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:43.462287 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:07:43.497986 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:43.498811 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:43.505402 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:43.506672 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:43.507699 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:43.508213 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:43.543487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-353d7760-8609-401a-9a0e-03da135d7ace tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "DELETE /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 204 len: 0 microversion: 2.1 time: 0.120931 Jul 24 01:07:43.544074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 440/874] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:07:43 2022] DELETE /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:07:43.553388 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:43.554111 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:43.564099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:43.564901 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:43.566601 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:43.567330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:43.718757 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:43.728643 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:43.730303 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:43.731597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bedaacde-a3a9-4fe7-aa80-7908841e1d78 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1722 microversion: 2.20 time: 0.238001 Jul 24 01:07:43.732416 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 435/875] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:43 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1722 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 24 01:07:43.751254 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:43.760248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e2e52690-3048-4852-8834-b3b8738d6a07 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1503 microversion: 2.1 time: 0.211712 Jul 24 01:07:43.760824 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 441/876] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:43 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1503 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:07:44.355407 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-5731d450-e90e-4bb6-a0f3-cd9835b54817 req-2a5544f2-e37c-4df2-a17f-4f1dc96c0173 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c70b3837-80be-480a-9a3f-7b03aff78573", "name": "network-vif-deleted", "tag": "1722886c-350d-4118-96b1-5082f64a6e70"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:44.363587 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-5731d450-e90e-4bb6-a0f3-cd9835b54817 req-2a5544f2-e37c-4df2-a17f-4f1dc96c0173 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:44.364036 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-5731d450-e90e-4bb6-a0f3-cd9835b54817 req-2a5544f2-e37c-4df2-a17f-4f1dc96c0173 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:44.375931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-5731d450-e90e-4bb6-a0f3-cd9835b54817 req-2a5544f2-e37c-4df2-a17f-4f1dc96c0173 service nova] Creating event network-vif-deleted:1722886c-350d-4118-96b1-5082f64a6e70 for instance c70b3837-80be-480a-9a3f-7b03aff78573 on hv-cinder-maste Jul 24 01:07:44.381699 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-5731d450-e90e-4bb6-a0f3-cd9835b54817 req-2a5544f2-e37c-4df2-a17f-4f1dc96c0173 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029423 Jul 24 01:07:44.382304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 436/877] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:44.750821 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:44.751602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:44.758575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:44.759154 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:44.759937 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:44.760379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:44.779038 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:44.780458 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:44.787221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:44.787816 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:44.788648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:44.789155 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:44.824328 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:44.833128 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:44.834708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:44.835968 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-4f3b103c-3644-4b53-aa40-ece9b0af4078 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1654 microversion: 2.20 time: 0.089270 Jul 24 01:07:44.836533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 442/878] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:44 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1654 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:44.936885 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:44.944841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2ed9ead8-3889-4a7c-97c0-60874f59ec08 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1503 microversion: 2.1 time: 0.170674 Jul 24 01:07:44.945434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 437/879] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:44 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1503 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:07:45.853658 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:45.854299 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:45.860835 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:45.861281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:45.862043 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:45.862480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:45.925423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:45.933790 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:45.935320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:45.936464 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-43ca6572-a9a7-46e4-abb7-32f2b9f8fde8 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1654 microversion: 2.20 time: 0.086891 Jul 24 01:07:45.937023 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 443/880] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:45 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1654 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:45.961410 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:45.962015 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:45.968623 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:45.969069 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:45.969830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:45.970266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:46.096962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:46.105111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3531ade2-c0e0-49e6-9c3c-9faecaffea33 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1503 microversion: 2.1 time: 0.147568 Jul 24 01:07:46.105675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 438/881] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:45 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1503 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:46.955575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:46.956157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:46.964227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:46.964674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:46.965432 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:46.965873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:47.029801 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:47.039161 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:47.041264 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '9a9256b6d7e24c54b061673cd17217cd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '75293bf0100f435bb567b690edc85533', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:47.042712 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c46cc792-3f52-4873-b526-9ab10b450bb3 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 200 len: 1365 microversion: 2.20 time: 0.091164 Jul 24 01:07:47.043364 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 444/882] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:46 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 1365 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 24 01:07:47.121675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:47.122350 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:47.129098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:47.129574 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:47.131137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:47.131581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:47.276830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:47.284561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-05b8d99a-506f-4941-aeec-81cfb363676f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1503 microversion: 2.1 time: 0.166693 Jul 24 01:07:47.285142 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 439/883] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:47 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1503 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:47.816788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-c07d1727-727a-4c43-9292-4f7c4012c5c2 req-9486c03b-766e-4a03-98c7-c0d93fd60283 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d56f41a6-596a-4045-9f75-d1972a1479f8", "name": "network-vif-deleted", "tag": "73b83928-1357-4598-a9a2-723487673e12"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:47.826505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-c07d1727-727a-4c43-9292-4f7c4012c5c2 req-9486c03b-766e-4a03-98c7-c0d93fd60283 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:47.826978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-c07d1727-727a-4c43-9292-4f7c4012c5c2 req-9486c03b-766e-4a03-98c7-c0d93fd60283 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:47.838751 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-c07d1727-727a-4c43-9292-4f7c4012c5c2 req-9486c03b-766e-4a03-98c7-c0d93fd60283 service nova] Creating event network-vif-deleted:73b83928-1357-4598-a9a2-723487673e12 for instance d56f41a6-596a-4045-9f75-d1972a1479f8 on hv-cinder-maste Jul 24 01:07:47.843499 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-c07d1727-727a-4c43-9292-4f7c4012c5c2 req-9486c03b-766e-4a03-98c7-c0d93fd60283 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030447 Jul 24 01:07:47.844078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 445/884] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:48.060484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:48.061012 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:48.067118 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:48.067576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:48.069307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:48.069727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:48.082849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] HTTP exception thrown: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Request method failure captured: Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 HTTP/1.1 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm9HYDgM9sKkcXoQzs8hJi4Lhvo-EPUoQwkjM0J25ggK_d-5LQx1tfWsBOgFVQxk8NaQ2JPzC3inqUcK2MCKp5-NaPPa10p1dI04lzI-IULSiFk043Gi5E6XOGZFE9jnTv8QQOR0z11fRq7Klx1D_JA_dbRl6IcJI-0eRqM7IltuAk Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Openstack-Nova-Api-Version: 2.20 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:48.084523 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75293bf0100f435bb567b690edc85533"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75293bf0100f435bb567b690edc85533", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 9a9256b6d7e24c54b061673cd17217cd Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:48.086237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'c70b3837-80be-480a-9a3f-7b03aff78573'} Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:48.087338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:48.088841 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:48.090273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Returning 404 to user: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:48.090273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3944f54f-3133-4fe3-b419-4a364eb0f562 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 404 len: 111 microversion: 2.20 time: 0.029450 Jul 24 01:07:48.090273 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 440/885] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:48 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 24 01:07:48.096238 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e591a4ed-b453-4230-96bb-c772077baf22 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '.version_select at 0x7f5e304a2d30>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:48.112953 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e591a4ed-b453-4230-96bb-c772077baf22 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-274235092" status: 204 len: 0 microversion: 2.20 time: 0.021413 Jul 24 01:07:48.113507 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 446/886] 10.222.0.18 () {60 vars in 1258 bytes} [Sun Jul 24 01:07:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-274235092 => generated 0 bytes in 23 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 24 01:07:48.301423 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:48.303531 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:48.311907 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:48.312379 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:48.313259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:48.313778 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:48.351240 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:48.351690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:48.352149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lazy-loading 'fault' on Instance uuid d56f41a6-596a-4045-9f75-d1972a1479f8 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:07:48.398059 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:48.404848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1966b6ab-7027-4a11-83c5-24bca30e7c86 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 200 len: 1303 microversion: 2.1 time: 0.107286 Jul 24 01:07:48.405626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 441/887] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:48 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 1303 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:49.424144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:49.424989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: d56f41a6-596a-4045-9f75-d1972a1479f8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:49.433129 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:49.433691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:49.434455 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:49.434923 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:49.449113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] HTTP exception thrown: Instance d56f41a6-596a-4045-9f75-d1972a1479f8 could not be found. Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Request method failure captured: Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 HTTP/1.1 Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3JtRyBUhzHhX-Kfbb4d7TFgyiftDm3CyYhkl9el2N7sR5OSb4EP6dyZeAvt1EvfJhoTFxwxC489c1vUhB6SG_UDHWqux9mTf1OQBgGLiWpnxSmkZQY8Yr15NPmmWv7Y-VudfXCc3xeysHzkoxvLKN84TOppjBwJyofmnBNEGbdYuu8k Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 54b31a874a6b4ec48f9a938d817c4637 Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-VolumesActionsTest-1112095850 Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:49.450714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/54b31a874a6b4ec48f9a938d817c4637"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/54b31a874a6b4ec48f9a938d817c4637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/54b31a874a6b4ec48f9a938d817c4637"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_54b31a874a6b4ec48f9a938d817c4637", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-VolumesActionsTest-1112095850 Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 54b31a874a6b4ec48f9a938d817c4637 Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-VolumesActionsTest-1112095850 Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-VolumesActionsTest-1112095850-project Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: ac64444a4b794026a5e95584d31f1c04 Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-VolumesActionsTest-1112095850-project Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance d56f41a6-596a-4045-9f75-d1972a1479f8 could not be found. Jul 24 01:07:49.452281 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'd56f41a6-596a-4045-9f75-d1972a1479f8'} Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d56f41a6-596a-4045-9f75-d1972a1479f8 could not be found. Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:49.453184 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d56f41a6-596a-4045-9f75-d1972a1479f8 could not be found. Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d56f41a6-596a-4045-9f75-d1972a1479f8 could not be found. Jul 24 01:07:49.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.455981 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Returning 404 to user: Instance d56f41a6-596a-4045-9f75-d1972a1479f8 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:49.455981 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f8d18da7-0a6b-4b37-8e6b-d86a85d981e0 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8" status: 404 len: 111 microversion: 2.1 time: 0.032997 Jul 24 01:07:49.455981 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 447/888] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:49 2022] GET /compute/v2.1/servers/d56f41a6-596a-4045-9f75-d1972a1479f8 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:07:49.603874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-567670102", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452", "networks": [{"uuid": "c741dacb-eb73-4391-9009-40d0df6ad4b9"}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:49.654842 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] validate_networks() for [('c741dacb-eb73-4391-9009-40d0df6ad4b9', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:07:49.883905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:49.884559 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:49.890618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:49.891173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:49.891950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:49.892390 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:49.905695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] HTTP exception thrown: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Request method failure captured: Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 HTTP/1.1 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm9HYDgM9sKkcXoQzs8hJi4Lhvo-EPUoQwkjM0J25ggK_d-5LQx1tfWsBOgFVQxk8NaQ2JPzC3inqUcK2MCKp5-NaPPa10p1dI04lzI-IULSiFk043Gi5E6XOGZFE9jnTv8QQOR0z11fRq7Klx1D_JA_dbRl6IcJI-0eRqM7IltuAk Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Openstack-Nova-Api-Version: 2.20 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:49.907283 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75293bf0100f435bb567b690edc85533"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75293bf0100f435bb567b690edc85533", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 9a9256b6d7e24c54b061673cd17217cd Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.908934 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'c70b3837-80be-480a-9a3f-7b03aff78573'} Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:49.909731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.911024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.913100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Returning 404 to user: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:49.913100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-837abdad-8ebf-4bc4-ac0c-e1700c3d2f3f tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 404 len: 111 microversion: 2.20 time: 0.028797 Jul 24 01:07:49.913100 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 448/889] 10.222.0.18 () {60 vars in 1279 bytes} [Sun Jul 24 01:07:49 2022] DELETE /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 24 01:07:49.917498 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:49.918099 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: c70b3837-80be-480a-9a3f-7b03aff78573] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:49.924106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:49.924498 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:49.925268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:49.925705 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:49.939691 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] HTTP exception thrown: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Request method failure captured: Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 HTTP/1.1 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm9HYDgM9sKkcXoQzs8hJi4Lhvo-EPUoQwkjM0J25ggK_d-5LQx1tfWsBOgFVQxk8NaQ2JPzC3inqUcK2MCKp5-NaPPa10p1dI04lzI-IULSiFk043Gi5E6XOGZFE9jnTv8QQOR0z11fRq7Klx1D_JA_dbRl6IcJI-0eRqM7IltuAk Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Openstack-Nova-Api-Version: 2.20 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:49.941252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75293bf0100f435bb567b690edc85533"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75293bf0100f435bb567b690edc85533", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 9a9256b6d7e24c54b061673cd17217cd Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.943143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': 'c70b3837-80be-480a-9a3f-7b03aff78573'} Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:49.944162 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. Jul 24 01:07:49.945413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:49.946488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Returning 404 to user: Instance c70b3837-80be-480a-9a3f-7b03aff78573 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:49.946488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ca8c3e94-2f01-49aa-bfe1-60a685c05c27 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573" status: 404 len: 111 microversion: 2.20 time: 0.028836 Jul 24 01:07:49.946488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 449/890] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:49 2022] GET /compute/v2.1/servers/c70b3837-80be-480a-9a3f-7b03aff78573 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 24 01:07:50.006973 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:07:50.007398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:07:50.007874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:07:50.008281 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:07:50.008808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:07:50.017943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:50.018393 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:50.019237 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:50.019671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:50.038262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Getting quotas for project 54b31a874a6b4ec48f9a938d817c4637. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:07:50.043626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Getting quotas for user ac64444a4b794026a5e95584d31f1c04 and project 54b31a874a6b4ec48f9a938d817c4637. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:07:50.056008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:07:50.058307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:07:50.101810 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-de72c252-a261-44de-8cc5-5b128b0ee765 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.502024 Jul 24 01:07:50.102449 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 442/891] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:07:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 503 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:07:50.112478 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ab71d730-0bfb-4253-9177-6a22c464760b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:50.113087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ab71d730-0bfb-4253-9177-6a22c464760b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:50.170026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ab71d730-0bfb-4253-9177-6a22c464760b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:50.177106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ab71d730-0bfb-4253-9177-6a22c464760b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1213 microversion: 2.1 time: 0.068341 Jul 24 01:07:50.178156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 450/892] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:50 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1213 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:51.086883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:51.087613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:51.095325 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:51.095892 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:51.096775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:51.097440 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:51.111640 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] HTTP exception thrown: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Request method failure captured: Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 HTTP/1.1 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jm9HYDgM9sKkcXoQzs8hJi4Lhvo-EPUoQwkjM0J25ggK_d-5LQx1tfWsBOgFVQxk8NaQ2JPzC3inqUcK2MCKp5-NaPPa10p1dI04lzI-IULSiFk043Gi5E6XOGZFE9jnTv8QQOR0z11fRq7Klx1D_JA_dbRl6IcJI-0eRqM7IltuAk Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Openstack-Nova-Api-Version: 2.20 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:07:51.113226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75293bf0100f435bb567b690edc85533"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75293bf0100f435bb567b690edc85533", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 9a9256b6d7e24c54b061673cd17217cd Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.114576 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '281e8695-c8e7-4293-8abe-a4f88f175084'} Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:51.115343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.117749 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.119974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Returning 404 to user: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:51.119974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fb2a22da-c094-4052-8f86-76fceca50f48 tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 404 len: 111 microversion: 2.20 time: 0.033855 Jul 24 01:07:51.119974 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 443/893] 10.222.0.18 () {60 vars in 1279 bytes} [Sun Jul 24 01:07:51 2022] DELETE /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 24 01:07:51.125084 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:51.125697 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] [instance: 281e8695-c8e7-4293-8abe-a4f88f175084] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:51.135040 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:51.135624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:51.136539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:51.137073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:51.161195 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] HTTP exception thrown: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Request method failure captured: Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 HTTP/1.1 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jm9HYDgM9sKkcXoQzs8hJi4Lhvo-EPUoQwkjM0J25ggK_d-5LQx1tfWsBOgFVQxk8NaQ2JPzC3inqUcK2MCKp5-NaPPa10p1dI04lzI-IULSiFk043Gi5E6XOGZFE9jnTv8QQOR0z11fRq7Klx1D_JA_dbRl6IcJI-0eRqM7IltuAk Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Openstack-Nova-Api-Version: 2.20 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:07:51.162958 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/75293bf0100f435bb567b690edc85533"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/75293bf0100f435bb567b690edc85533"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_75293bf0100f435bb567b690edc85533", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: 75293bf0100f435bb567b690edc85533 Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1026315772 Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 9a9256b6d7e24c54b061673cd17217cd Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1026315772-project Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.164594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '281e8695-c8e7-4293-8abe-a4f88f175084'} Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:07:51.165520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. Jul 24 01:07:51.167153 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:07:51.168831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] Returning 404 to user: Instance 281e8695-c8e7-4293-8abe-a4f88f175084 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:07:51.168831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c95b0ffa-6427-4606-83c1-a0ec20be777e tempest-AttachVolumeShelveTestJSON-1026315772 tempest-AttachVolumeShelveTestJSON-1026315772-project] 10.222.0.18 "GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084" status: 404 len: 111 microversion: 2.20 time: 0.046136 Jul 24 01:07:51.168831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 451/894] 10.222.0.18 () {60 vars in 1276 bytes} [Sun Jul 24 01:07:51 2022] GET /compute/v2.1/servers/281e8695-c8e7-4293-8abe-a4f88f175084 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 24 01:07:51.194904 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:51.195641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:51.202956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:51.203471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:51.204594 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:51.205202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:51.277262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:51.286519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0a541d47-c998-400d-97d6-c915c321f936 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1271 microversion: 2.1 time: 0.096889 Jul 24 01:07:51.288054 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 444/895] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:51 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1271 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:52.305480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:52.306836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:52.313876 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:52.314584 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:52.316149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:52.316758 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:52.508447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-1216bccc-dd2c-46bf-9735-0e3feac7ee38 req-88ca5587-c3e2-45ed-b56e-e7e3979e8c87 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f8c1e487-f402-4b2a-b202-4122298b04ca", "tag": "feb2b0f6-e2a4-4f57-9d0f-6b4477fb5910"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:52.522564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-1216bccc-dd2c-46bf-9735-0e3feac7ee38 req-88ca5587-c3e2-45ed-b56e-e7e3979e8c87 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:52.523464 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-1216bccc-dd2c-46bf-9735-0e3feac7ee38 req-88ca5587-c3e2-45ed-b56e-e7e3979e8c87 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:52.536239 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-1216bccc-dd2c-46bf-9735-0e3feac7ee38 req-88ca5587-c3e2-45ed-b56e-e7e3979e8c87 service nova] Creating event network-changed:feb2b0f6-e2a4-4f57-9d0f-6b4477fb5910 for instance f8c1e487-f402-4b2a-b202-4122298b04ca on hv-cinder-maste Jul 24 01:07:52.541373 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-1216bccc-dd2c-46bf-9735-0e3feac7ee38 req-88ca5587-c3e2-45ed-b56e-e7e3979e8c87 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043197 Jul 24 01:07:52.542304 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 445/896] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:07:52.599198 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:52.607633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-aef50429-e722-4704-b3ea-9b70a1a55701 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.306715 Jul 24 01:07:52.608404 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 452/897] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:52 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:53.626263 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:53.627026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:53.634327 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:53.634792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:53.635564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:53.635997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:53.929258 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:53.939092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-90bd1e42-d1e4-4bf4-b00a-9a33517961ab tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.317535 Jul 24 01:07:53.939874 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 446/898] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:53 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:54.958233 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:54.958926 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:54.966222 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:54.966803 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:54.967651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:54.968157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:54.987907 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:07:55.126251 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:55.134434 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a177bc2c-d2e7-454e-b2bc-6e316d43267c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.180640 Jul 24 01:07:55.135038 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 453/899] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:54 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:56.152300 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:56.152945 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:56.159927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:56.160381 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:56.161158 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:56.161593 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:56.403208 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:56.411480 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-508c939e-622c-492c-994d-636494a79c85 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.263222 Jul 24 01:07:56.413538 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 447/900] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:56 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:57.429818 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:57.430623 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:57.437402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:57.437860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:57.438644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:57.439119 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:57.627519 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:57.637863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c68004d2-8c90-4dca-b6b1-c849ff529a26 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.212112 Jul 24 01:07:57.649182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 454/901] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:57 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:58.657849 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:58.658459 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:58.665709 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:58.666156 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:58.666952 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:58.667396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:58.860926 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:07:58.871223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e28f58d2-f224-4b9e-a2b7-476b3cef1b58 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.218813 Jul 24 01:07:58.872245 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 448/902] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:58 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:07:58.978610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-636e8b80-5425-4e31-9d77-8e970c28cfe7 req-fab2d4f1-2c4f-4b3b-9cf0-ea551a954a70 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8c1e487-f402-4b2a-b202-4122298b04ca", "name": "network-vif-plugged", "status": "completed", "tag": "feb2b0f6-e2a4-4f57-9d0f-6b4477fb5910"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:07:58.987998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-636e8b80-5425-4e31-9d77-8e970c28cfe7 req-fab2d4f1-2c4f-4b3b-9cf0-ea551a954a70 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:58.988438 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-636e8b80-5425-4e31-9d77-8e970c28cfe7 req-fab2d4f1-2c4f-4b3b-9cf0-ea551a954a70 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:59.001558 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-636e8b80-5425-4e31-9d77-8e970c28cfe7 req-fab2d4f1-2c4f-4b3b-9cf0-ea551a954a70 service nova] Creating event network-vif-plugged:feb2b0f6-e2a4-4f57-9d0f-6b4477fb5910 for instance f8c1e487-f402-4b2a-b202-4122298b04ca on hv-cinder-maste Jul 24 01:07:59.006563 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-636e8b80-5425-4e31-9d77-8e970c28cfe7 req-fab2d4f1-2c4f-4b3b-9cf0-ea551a954a70 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032615 Jul 24 01:07:59.007188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 455/903] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:07:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:07:59.888422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:07:59.889026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:07:59.895353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:59.895808 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:07:59.897140 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:07:59.897693 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:00.026467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:00.050943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:00.076307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:00.083633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f265fba8-0e0e-4716-8a45-012f68128edb tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1313 microversion: 2.1 time: 0.199430 Jul 24 01:08:00.084721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 449/904] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:07:59 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1313 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:01.099373 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:01.102330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:01.103085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:01.109905 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:01.110288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:01.111083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:01.111519 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:01.247255 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:01.254721 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-54a0ffc6-1ddc-415d-8eaa-d6cd8807a988 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1497 microversion: 2.1 time: 0.158622 Jul 24 01:08:01.255309 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 456/905] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:01 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:01.774649 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:01.775176 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:01.783286 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:01.783916 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:01.785042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:01.785558 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:01.815567 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:08:01.900943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-156542c6-b7ee-49f7-9a82-60ce3297eab1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "DELETE /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 204 len: 0 microversion: 2.1 time: 0.130151 Jul 24 01:08:01.904908 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 450/906] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:08:01 2022] DELETE /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:08:01.911899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:01.912719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:01.922720 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:01.923060 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:01.923848 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:01.924291 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:02.083650 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:02.093082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a570c954-2d45-4190-b5e6-0b053699e92b tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1503 microversion: 2.1 time: 0.186662 Jul 24 01:08:02.093584 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 457/907] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:01 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1503 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:03.112442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:03.113146 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:03.119791 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:03.120251 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:03.121022 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:03.121462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:03.259291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:03.266803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2296fd41-a854-4df9-afa9-e4c3ca69bb8f tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1503 microversion: 2.1 time: 0.158551 Jul 24 01:08:03.267379 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 451/908] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:03 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1503 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:04.284054 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:04.284886 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:04.291450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:04.291906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:04.292678 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:04.293119 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:04.437236 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:04.446264 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-122f90c0-675b-4aa3-937c-b9ca71c7543c tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1503 microversion: 2.1 time: 0.166152 Jul 24 01:08:04.446851 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 458/909] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:04 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1503 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:05.463566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:05.464182 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:05.476050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:05.476504 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:05.477288 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:05.477733 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:05.480600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:05.618292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:05.625789 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-2fe5ac21-2971-4a5d-ba11-d7c02d402ae1 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1503 microversion: 2.1 time: 0.166257 Jul 24 01:08:05.626363 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 452/910] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:05 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1503 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:06.644125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:06.644872 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:06.652423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:06.653190 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:06.653953 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:06.654385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:06.796873 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:06.805828 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9673b01b-e2fa-468e-98ad-fdb401c94b8e tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1503 microversion: 2.1 time: 0.166127 Jul 24 01:08:06.806492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 459/911] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:06 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1503 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:07.825267 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:07.826039 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:07.833396 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:07.834290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:07.835668 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:07.836184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:08.002411 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac64444a4b794026a5e95584d31f1c04', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54b31a874a6b4ec48f9a938d817c4637', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:08.010322 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4a23e0b1-f2f7-4e2d-8d6a-6f5d1b445fb5 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 200 len: 1503 microversion: 2.1 time: 0.191284 Jul 24 01:08:08.019796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 453/912] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:07 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 1503 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:08:08.220643 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-c2ad04fe-336a-4bc8-9781-3d5d209344e9 req-81827b88-95f0-4a6a-9e5c-5610c6c2f0f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8c1e487-f402-4b2a-b202-4122298b04ca", "name": "network-vif-deleted", "tag": "feb2b0f6-e2a4-4f57-9d0f-6b4477fb5910"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:08:08.232589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-c2ad04fe-336a-4bc8-9781-3d5d209344e9 req-81827b88-95f0-4a6a-9e5c-5610c6c2f0f9 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:08.233040 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-c2ad04fe-336a-4bc8-9781-3d5d209344e9 req-81827b88-95f0-4a6a-9e5c-5610c6c2f0f9 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:08.248012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-c2ad04fe-336a-4bc8-9781-3d5d209344e9 req-81827b88-95f0-4a6a-9e5c-5610c6c2f0f9 service nova] Creating event network-vif-deleted:feb2b0f6-e2a4-4f57-9d0f-6b4477fb5910 for instance f8c1e487-f402-4b2a-b202-4122298b04ca on hv-cinder-maste Jul 24 01:08:08.254753 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-c2ad04fe-336a-4bc8-9781-3d5d209344e9 req-81827b88-95f0-4a6a-9e5c-5610c6c2f0f9 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037701 Jul 24 01:08:08.256005 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 460/913] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:08:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:08:09.028087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:09.028734 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] [instance: f8c1e487-f402-4b2a-b202-4122298b04ca] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:09.035471 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:09.035940 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:09.036724 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:09.037167 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:09.051074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] HTTP exception thrown: Instance f8c1e487-f402-4b2a-b202-4122298b04ca could not be found. Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Request method failure captured: Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca HTTP/1.1 Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JtRyBUhzHhX-Kfbb4d7TFgyiftDm3CyYhkl9el2N7sR5OSb4EP6dyZeAvt1EvfJhoTFxwxC489c1vUhB6SG_UDHWqux9mTf1OQBgGLiWpnxSmkZQY8Yr15NPmmWv7Y-VudfXCc3xeysHzkoxvLKN84TOppjBwJyofmnBNEGbdYuu8k Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 54b31a874a6b4ec48f9a938d817c4637 Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesActionsTest-1112095850 Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:08:09.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/54b31a874a6b4ec48f9a938d817c4637"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/54b31a874a6b4ec48f9a938d817c4637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/54b31a874a6b4ec48f9a938d817c4637"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_54b31a874a6b4ec48f9a938d817c4637", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesActionsTest-1112095850 Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 54b31a874a6b4ec48f9a938d817c4637 Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesActionsTest-1112095850 Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesActionsTest-1112095850-project Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: ac64444a4b794026a5e95584d31f1c04 Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesActionsTest-1112095850-project Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance f8c1e487-f402-4b2a-b202-4122298b04ca could not be found. Jul 24 01:08:09.054367 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'f8c1e487-f402-4b2a-b202-4122298b04ca'} Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8c1e487-f402-4b2a-b202-4122298b04ca could not be found. Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:08:09.055506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8c1e487-f402-4b2a-b202-4122298b04ca could not be found. Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8c1e487-f402-4b2a-b202-4122298b04ca could not be found. Jul 24 01:08:09.057035 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:08:09.058434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] Returning 404 to user: Instance f8c1e487-f402-4b2a-b202-4122298b04ca could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:08:09.058434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bdbc13cf-5436-491a-8df5-8f23e1c2a193 tempest-VolumesActionsTest-1112095850 tempest-VolumesActionsTest-1112095850-project] 10.222.0.18 "GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca" status: 404 len: 111 microversion: 2.1 time: 0.029893 Jul 24 01:08:09.058434 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 454/914] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:09 2022] GET /compute/v2.1/servers/f8c1e487-f402-4b2a-b202-4122298b04ca => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:08:37.890626 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-967614601", "imageRef": "b1307b49-a380-4fe2-8bdd-cdd9016ef4b0", "flavorRef": "452"}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:08:37.896151 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:37.897538 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=36 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:37.945120 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] validate_networks() for None {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:08:38.220326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:08:38.220812 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:08:38.221331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:08:38.221748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:08:38.222167 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:08:38.231972 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:38.232424 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:38.233229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:38.233671 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:38.250595 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Getting quotas for project affbcb4611414dcba4e3eca7c0686be7. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:08:38.254617 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Getting quotas for user 4707858b7ef84a7fbd74a93b91cc6e68 and project affbcb4611414dcba4e3eca7c0686be7. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:08:38.266065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:08:38.268411 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b1307b49-a380-4fe2-8bdd-cdd9016ef4b0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:08:38.310664 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1aeba52d-14bf-4db4-b908-ba4fdfc12fe6 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.424321 Jul 24 01:08:38.311346 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 461/915] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:08:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 426 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:08:38.322425 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0a9380db-f77f-4565-928f-cab2332c008b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:38.323062 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0a9380db-f77f-4565-928f-cab2332c008b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:38.325102 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:38.325939 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=13 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:38.382550 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0a9380db-f77f-4565-928f-cab2332c008b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:38.389404 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0a9380db-f77f-4565-928f-cab2332c008b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1214 microversion: 2.1 time: 0.071783 Jul 24 01:08:38.389952 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 455/916] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:38 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1214 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:39.406081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:39.406686 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:39.413271 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:39.413716 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:39.414473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:39.414950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:39.477534 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:39.484941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-41a441ce-56f1-46f7-bfaf-414e9079c73f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1274 microversion: 2.1 time: 0.083370 Jul 24 01:08:39.485495 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 462/917] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:39 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1274 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:40.502641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:40.503424 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:40.510230 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:40.510684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:40.511508 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:40.511943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:40.588847 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:40.596626 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7c9d3161-4ca6-4c32-83cc-35365c0a8a52 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1274 microversion: 2.1 time: 0.097963 Jul 24 01:08:40.597448 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 456/918] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:40 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1274 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:41.613400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:41.614011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:41.620776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:41.621231 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:41.622001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:41.622435 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:41.687827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:41.697472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a85481fd-873f-408b-98d4-fc059971008f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1274 microversion: 2.1 time: 0.087902 Jul 24 01:08:41.698364 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 463/919] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:41 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1274 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:42.713568 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:42.715098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:42.723014 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:42.723913 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:42.726070 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:42.726798 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:42.830266 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:42.841717 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d65bbe31-93c2-40bc-a00a-8bc6e2945c69 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1272 microversion: 2.1 time: 0.131770 Jul 24 01:08:42.843439 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 457/920] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:42 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1272 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:43.861301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:43.861920 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:43.869641 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:43.870406 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:43.871377 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:43.871989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:44.137176 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:44.145665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-940d68a0-1a0e-44ee-9f28-b23ae24e6a8a tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.288287 Jul 24 01:08:44.147087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 464/921] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:43 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:45.163894 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:45.165330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:45.172226 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:45.172933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:45.173883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:45.175324 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:45.426578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:45.456714 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4f738b6a-47d5-4350-a9d9-b9edc09f32e2 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.297029 Jul 24 01:08:45.457848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 458/922] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:45 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:46.368571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-76ed0c18-3586-4605-9157-2daf54958828 req-d3f1d30a-bea6-4eff-94b3-584118be24b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29913286-c7e7-4015-b3d2-dd1bd488669a", "tag": "0b710361-9202-4663-9466-c4fbc8c08591"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:08:46.389127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-76ed0c18-3586-4605-9157-2daf54958828 req-d3f1d30a-bea6-4eff-94b3-584118be24b1 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:46.395054 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-76ed0c18-3586-4605-9157-2daf54958828 req-d3f1d30a-bea6-4eff-94b3-584118be24b1 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:46.420072 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-76ed0c18-3586-4605-9157-2daf54958828 req-d3f1d30a-bea6-4eff-94b3-584118be24b1 service nova] Creating event network-changed:0b710361-9202-4663-9466-c4fbc8c08591 for instance 29913286-c7e7-4015-b3d2-dd1bd488669a on hv-cinder-maste Jul 24 01:08:46.424837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-76ed0c18-3586-4605-9157-2daf54958828 req-d3f1d30a-bea6-4eff-94b3-584118be24b1 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062525 Jul 24 01:08:46.426006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 465/923] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:08:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:08:46.474989 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:46.476732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:46.494192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:46.494962 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:46.495975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:46.496633 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:46.684528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:46.694098 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6cbe98f0-32c6-4d0f-8339-d76c1b3dc97c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.222247 Jul 24 01:08:46.695063 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 459/924] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:46 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:47.713083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:47.713891 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:47.720596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:47.721385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:47.723224 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:47.723913 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:47.966609 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:47.975019 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7bf06b9c-817d-4b83-a618-b7aaaf57ebb4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.266987 Jul 24 01:08:47.975651 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 466/925] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:47 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:48.994882 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:48.996269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:49.004444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:49.006851 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:49.007528 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:49.008721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:49.010519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:49.830794 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:49.846191 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fb7e1402-f4db-4ac0-91fb-f0dedd661a44 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.857991 Jul 24 01:08:49.847385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 460/926] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:48 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 860 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:50.865357 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:50.866642 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:50.877858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:50.878937 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:08:50.884452 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:50.885265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:50.886872 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:50.888191 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:51.408490 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:51.423796 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d37ab6ca-4a1e-4a29-88ca-efbd64126b06 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.559201 Jul 24 01:08:51.425883 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 467/927] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:50 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:52.443690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:52.445024 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:52.456676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:52.457487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:52.459360 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:52.460080 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:52.642802 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:52.652943 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3be41a95-2aad-481d-87b2-64ab7681f3e0 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.215705 Jul 24 01:08:52.653530 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 461/928] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:52 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:52.723423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2ff80516-0856-4fbe-93ff-fa0c627330bc tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Action: 'create', calling method: .version_select at 0x7f5e304a5c10>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1886212345", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOnrtOXfahuAvyCjRwRJ0Ak5mPiVxjUXO14f4oUujrWrL13jSdPbeQA4bCaXfzl3SBj9TASRUYJ17WlWJJAT6alX6tNNF/qEiSy1SRIzBrQ4nbmjM/KV+Ip8aTJ7IAZh8w=="}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:08:52.738085 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-2ff80516-0856-4fbe-93ff-fa0c627330bc tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for project be64d6ea0f0149608c9a823425a0e811. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:08:52.741925 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-2ff80516-0856-4fbe-93ff-fa0c627330bc tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for user 7c545f7be7794566b639c29c0717c17a and project be64d6ea0f0149608c9a823425a0e811. Resources: {'key_pairs'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:08:52.764208 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2ff80516-0856-4fbe-93ff-fa0c627330bc tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.297257 Jul 24 01:08:52.764917 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 468/929] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:08:52 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:08:53.671030 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:53.671690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:53.688110 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:53.688580 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:53.689611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:53.690417 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:53.791997 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-2e137de2-a760-4aaf-85a6-439350975585 req-329c6675-d408-4163-ae70-8498ca8523ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29913286-c7e7-4015-b3d2-dd1bd488669a", "name": "network-vif-plugged", "status": "completed", "tag": "0b710361-9202-4663-9466-c4fbc8c08591"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:08:53.805517 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2e137de2-a760-4aaf-85a6-439350975585 req-329c6675-d408-4163-ae70-8498ca8523ef service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:53.806020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-2e137de2-a760-4aaf-85a6-439350975585 req-329c6675-d408-4163-ae70-8498ca8523ef service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:53.825657 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-2e137de2-a760-4aaf-85a6-439350975585 req-329c6675-d408-4163-ae70-8498ca8523ef service nova] Creating event network-vif-plugged:0b710361-9202-4663-9466-c4fbc8c08591 for instance 29913286-c7e7-4015-b3d2-dd1bd488669a on hv-cinder-maste Jul 24 01:08:53.835722 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-2e137de2-a760-4aaf-85a6-439350975585 req-329c6675-d408-4163-ae70-8498ca8523ef service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044739 Jul 24 01:08:53.838295 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 469/930] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:08:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:08:53.881939 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:53.889707 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3f12b974-4f0b-4ab1-8b5e-3c3c6307e66c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.223089 Jul 24 01:08:53.890541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 462/931] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:53 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:54.909068 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:54.909718 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:54.916974 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:54.917610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:54.918513 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:54.919157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:55.075183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:55.082759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8201c3ff-1a5c-4381-9ccc-752776c83a45 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.178968 Jul 24 01:08:55.083877 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 470/932] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:54 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:56.101959 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:56.102581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:56.110449 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:56.110927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:56.111739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:56.112147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:56.256196 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:56.264005 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-63a34696-676c-4ff4-a6ce-d3d4bd64ec93 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.166710 Jul 24 01:08:56.264602 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 463/933] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:56 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:57.283035 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:57.283646 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:57.290057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:57.290508 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:57.291596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:57.292129 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:57.432279 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:57.440158 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ce882267-984b-4886-bc39-d33b63617d27 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1314 microversion: 2.1 time: 0.161751 Jul 24 01:08:57.440816 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 471/934] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:57 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:58.457516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:58.458292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:58.465519 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:58.465999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:58.466852 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:58.467262 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:58.602247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:58.610123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7c07875d-5403-4040-a4c9-5e94ff88528b tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1464 microversion: 2.1 time: 0.156646 Jul 24 01:08:58.610697 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 464/935] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:58 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1464 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:58.653226 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:58.653824 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:58.661803 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:58.662326 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:58.663143 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:58.663584 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:58.690090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:08:58.780562 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-042a834c-43df-4c64-b467-1baee037c34f tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "DELETE /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 204 len: 0 microversion: 2.1 time: 0.130969 Jul 24 01:08:58.781188 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 472/936] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:08:58 2022] DELETE /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:08:58.790130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:58.790742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:58.796830 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:58.797283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:58.798049 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:58.798490 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:58.929491 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:08:58.937197 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fbcf8732-7951-46eb-a19c-15349e0821a4 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1470 microversion: 2.1 time: 0.151487 Jul 24 01:08:58.937788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 465/937] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:58 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1470 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:08:59.955868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:08:59.956515 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:08:59.963137 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:59.963592 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:08:59.964359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:08:59.964798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:00.099933 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:00.107644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-18c2e26f-5a44-4724-b68b-7d3d58b1b12c tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1470 microversion: 2.1 time: 0.156230 Jul 24 01:09:00.108214 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 473/938] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:08:59 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1470 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:01.125046 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:01.125613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:01.131915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:01.132369 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:01.133137 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:01.133581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:01.272923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:01.280792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e00558a0-a081-409f-aa48-82be7faa2f41 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1470 microversion: 2.1 time: 0.159764 Jul 24 01:09:01.281711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 466/939] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:01 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1470 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:02.299231 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:02.299853 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:02.307315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:02.307765 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:02.308815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:02.309243 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:02.445109 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:02.453080 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ee35f963-e123-4af2-b0fa-57c11a35d12d tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1470 microversion: 2.1 time: 0.157742 Jul 24 01:09:02.453662 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 474/940] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:02 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1470 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:03.471110 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-d4b6d98c-f212-44d1-bd75-44e26b3db99d req-55a42b61-e1da-44f3-aaab-db6a8525b8f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29913286-c7e7-4015-b3d2-dd1bd488669a", "name": "network-vif-deleted", "tag": "0b710361-9202-4663-9466-c4fbc8c08591"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:03.471918 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:03.472591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:03.480761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=4 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:03.481236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-d4b6d98c-f212-44d1-bd75-44e26b3db99d req-55a42b61-e1da-44f3-aaab-db6a8525b8f7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:03.481920 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-d4b6d98c-f212-44d1-bd75-44e26b3db99d req-55a42b61-e1da-44f3-aaab-db6a8525b8f7 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:03.488693 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:03.490149 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:03.491211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:03.491899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:03.497295 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-d4b6d98c-f212-44d1-bd75-44e26b3db99d req-55a42b61-e1da-44f3-aaab-db6a8525b8f7 service nova] Creating event network-vif-deleted:0b710361-9202-4663-9466-c4fbc8c08591 for instance 29913286-c7e7-4015-b3d2-dd1bd488669a on hv-cinder-maste Jul 24 01:09:03.502362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-d4b6d98c-f212-44d1-bd75-44e26b3db99d req-55a42b61-e1da-44f3-aaab-db6a8525b8f7 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035066 Jul 24 01:09:03.502999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 467/941] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:03.591318 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4707858b7ef84a7fbd74a93b91cc6e68', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'affbcb4611414dcba4e3eca7c0686be7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:03.605518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5f513a81-d77d-48ea-a0ca-87dd6bad8510 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 200 len: 1428 microversion: 2.1 time: 0.136822 Jul 24 01:09:03.606893 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 475/942] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:03 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 1428 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:04.625380 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:04.626092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] [instance: 29913286-c7e7-4015-b3d2-dd1bd488669a] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:04.632307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:04.632755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:04.633522 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:04.633958 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:04.650698 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] HTTP exception thrown: Instance 29913286-c7e7-4015-b3d2-dd1bd488669a could not be found. Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Request method failure captured: Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a HTTP/1.1 Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3JuUyQidf2SilgSkftZ5geBYvqNgIWUQZ29XNO-6LKiV3e68Ge4hswUA6ca_p6-10ZVLFss15bKas_4wuVhBoY7IMvsu1xeOBtGPOIQ6Y602SPdABu8Tt5qnTHG-HhGa9kAQkJ6JjtRrudZTz90o0kHRJD9wtvQFnicWECMS1Zd1mfg Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: affbcb4611414dcba4e3eca7c0686be7 Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-VolumesNegativeTest-1583220652 Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:09:04.652370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/affbcb4611414dcba4e3eca7c0686be7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/affbcb4611414dcba4e3eca7c0686be7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/affbcb4611414dcba4e3eca7c0686be7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_affbcb4611414dcba4e3eca7c0686be7", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-VolumesNegativeTest-1583220652 Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: affbcb4611414dcba4e3eca7c0686be7 Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-VolumesNegativeTest-1583220652 Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-VolumesNegativeTest-1583220652-project Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 4707858b7ef84a7fbd74a93b91cc6e68 Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-VolumesNegativeTest-1583220652-project Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 29913286-c7e7-4015-b3d2-dd1bd488669a could not be found. Jul 24 01:09:04.654114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '29913286-c7e7-4015-b3d2-dd1bd488669a'} Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29913286-c7e7-4015-b3d2-dd1bd488669a could not be found. Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:09:04.655044 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29913286-c7e7-4015-b3d2-dd1bd488669a could not be found. Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29913286-c7e7-4015-b3d2-dd1bd488669a could not be found. Jul 24 01:09:04.656383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:09:04.657546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] Returning 404 to user: Instance 29913286-c7e7-4015-b3d2-dd1bd488669a could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:09:04.657546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-52358ae9-8178-4cae-aea2-4445b28b5689 tempest-VolumesNegativeTest-1583220652 tempest-VolumesNegativeTest-1583220652-project] 10.222.0.18 "GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a" status: 404 len: 111 microversion: 2.1 time: 0.033443 Jul 24 01:09:04.657546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 468/943] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:04 2022] GET /compute/v2.1/servers/29913286-c7e7-4015-b3d2-dd1bd488669a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:09:04.663032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:09:04.663188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:09:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:09:11.674330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-681052029", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1886212345", "security_groups": [{"name": "tempest-secgroup-smoke-1806918878"}], "block_device_mapping_v2": [{"uuid": "9be425e4-32f5-49d6-91be-7bd6dbc5de73", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1b1fc3ab-4643-4369-871d-3a6b06dcc201"}]}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:11.683906 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:11.882838 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] validate_networks() for [('1b1fc3ab-4643-4369-871d-3a6b06dcc201', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:09:12.983090 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:09:12.986220 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:09:12.987577 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:09:12.988827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:09:12.990167 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:09:13.008591 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:13.010057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:13.011135 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:13.012004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:13.025019 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:13.049745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for project be64d6ea0f0149608c9a823425a0e811. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:09:13.055078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for user 7c545f7be7794566b639c29c0717c17a and project be64d6ea0f0149608c9a823425a0e811. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:09:13.141395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:09:13.143771 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='9be425e4-32f5-49d6-91be-7bd6dbc5de73',volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:09:13.354853 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8d76cfc5-c24b-4259-bf2e-662f4fd2e412 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.684244 Jul 24 01:09:13.357199 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 476/944] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:09:11 2022] POST /compute/v2.1/servers => generated 406 bytes in 1687 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:09:13.367348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5c6d1e30-f462-4fe5-b843-a59afb5f8d1e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:13.369268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5c6d1e30-f462-4fe5-b843-a59afb5f8d1e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:13.372736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=31 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:13.484008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5c6d1e30-f462-4fe5-b843-a59afb5f8d1e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:13.494176 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5c6d1e30-f462-4fe5-b843-a59afb5f8d1e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1094 microversion: 2.1 time: 0.131183 Jul 24 01:09:13.498018 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 469/945] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:13 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1094 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:09:14.512674 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:14.513344 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:14.529628 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:14.530596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:14.531493 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:14.532001 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:14.638351 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:14.651631 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e38abd8e-7293-4e1f-8568-95d38df8c9d3 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1192 microversion: 2.1 time: 0.143737 Jul 24 01:09:14.662840 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 477/946] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:14 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1192 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:15.669426 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:15.670119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:15.676906 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=70 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:15.693832 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:15.694278 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:15.696152 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:15.697815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:15.964815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:15.977228 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8eb1fddc-7715-4788-bc53-d6030bd4c60b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1278 microversion: 2.1 time: 0.311848 Jul 24 01:09:15.979296 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 470/947] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:15 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1278 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:16.624330 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-5862093a-eb2e-4dd5-b4b3-f5e4767497e9 req-b2c29b40-7504-4e28-87bd-7c00a23cda80 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "863bc791-b8b0-4b49-bf36-6dd4ab8b039d", "tag": "2dace519-865d-4dd1-84e9-96672860e4be"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:16.641565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-5862093a-eb2e-4dd5-b4b3-f5e4767497e9 req-b2c29b40-7504-4e28-87bd-7c00a23cda80 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:16.642019 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-5862093a-eb2e-4dd5-b4b3-f5e4767497e9 req-b2c29b40-7504-4e28-87bd-7c00a23cda80 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:16.669595 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-5862093a-eb2e-4dd5-b4b3-f5e4767497e9 req-b2c29b40-7504-4e28-87bd-7c00a23cda80 service nova] Creating event network-changed:2dace519-865d-4dd1-84e9-96672860e4be for instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d on hv-cinder-maste Jul 24 01:09:16.676071 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-5862093a-eb2e-4dd5-b4b3-f5e4767497e9 req-b2c29b40-7504-4e28-87bd-7c00a23cda80 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057981 Jul 24 01:09:16.681758 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 478/948] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:16.997252 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:16.998188 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:17.007286 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:17.008142 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:17.010564 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:17.011291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:17.280133 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:17.290758 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-fa5d90a7-2c4d-4b4b-b5bf-2ddb33a8a931 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.298029 Jul 24 01:09:17.291870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 471/949] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:16 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:18.313656 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:18.314394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:18.322328 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:18.322799 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:18.323659 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:18.324098 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:18.587533 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:18.598008 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ef6d9d7d-8ad6-4907-939a-d364135b45c6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.292683 Jul 24 01:09:18.610460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 479/950] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:18 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:19.615581 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:19.616199 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:19.624815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:19.625660 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:19.626654 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:19.629641 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:20.091053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:20.099371 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c7425bd6-a66b-4eae-926a-2b280392e2f5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.487947 Jul 24 01:09:20.104578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 472/951] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:19 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:21.121385 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:21.122009 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:21.130618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:21.132010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:21.132986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:21.135055 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:21.380692 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:21.401293 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-766e2581-0498-4405-9542-0a4cef5f908f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.283907 Jul 24 01:09:21.401884 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 480/952] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:21 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:21.735474 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3f2ee6f0-7277-4bee-9891-c95c42894bc6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Action: 'create', calling method: .version_select at 0x7f5e3023e820>, body: {"keypair": {"name": "tempest-TestShelveInstance-319784191", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPIwMtmHwgrJw79MQU6MPFvNYw4J30N23GCPHZF2oC84vwdm8Gu9rVV+N1mVeSsDwldEud0gA4mwk8u3qSfmFpNsiM3lkpumz+EkIQ8UXO9B2JT3Ng3xaywOote1i3YhoQ=="}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:21.749563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-3f2ee6f0-7277-4bee-9891-c95c42894bc6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Getting quotas for project 6af7b896ae8f4e09957a342a4c0d3030. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:09:21.753573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-3f2ee6f0-7277-4bee-9891-c95c42894bc6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Getting quotas for user b7a89576949f471f9cd92bbc6677175f and project 6af7b896ae8f4e09957a342a4c0d3030. Resources: {'key_pairs'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:09:21.780490 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3f2ee6f0-7277-4bee-9891-c95c42894bc6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.237261 Jul 24 01:09:21.781100 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 473/953] 10.222.0.18 () {60 vars in 1158 bytes} [Sun Jul 24 01:09:21 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:22.420542 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:22.421339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:22.427351 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:22.427803 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:22.428575 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:22.429012 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:22.607850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:22.616350 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fda41cf8-1e46-4620-897c-606171ceec20 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.201894 Jul 24 01:09:22.616944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 481/954] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:22 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:23.634752 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:23.635413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:23.643283 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:23.644117 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:23.645209 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:23.645651 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:23.835479 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:23.843527 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3a339437-3d14-4e9f-bbb6-f461b21f79ca tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.213880 Jul 24 01:09:23.844115 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 474/955] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:23 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:24.861032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:24.861743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:24.863858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:24.864582 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:24.876119 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:24.876576 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:24.877832 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:24.878280 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:25.016785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:25.024817 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1bd27fde-1019-45b5-ade9-732cb54cef0b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.167753 Jul 24 01:09:25.025382 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 482/956] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:24 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:26.042427 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:26.043120 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:26.050975 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:26.051507 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:26.052292 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:26.052734 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:26.192989 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:26.202090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9febc5fd-cea6-43b6-b874-5c8f74966b41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.164442 Jul 24 01:09:26.202675 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 475/957] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:26 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:27.219091 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:27.219821 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:27.227456 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:27.227992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:27.228779 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:27.229214 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:27.370941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:27.378351 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-82641c28-dfaf-4133-9e12-e99e90d73112 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.163346 Jul 24 01:09:27.378941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 483/958] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:27 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:28.395771 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:28.396379 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:28.403912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:28.404355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:28.405119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:28.405552 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:28.550009 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:28.557727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bcc62fcf-5eb4-4725-8ef0-e9650ff59451 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.165857 Jul 24 01:09:28.558287 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 476/959] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:28 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:29.575813 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:29.576422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:29.582967 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:29.583413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:29.584173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:29.584602 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:29.721390 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:29.729508 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e7e5f3a3-e310-4c86-b38b-b75bd1d291a6 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.159120 Jul 24 01:09:29.730067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 484/960] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:29 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:30.748143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:30.748681 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:30.754664 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:30.755149 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:30.755915 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:30.756357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:30.893001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:30.902342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9db42edb-23a8-4c71-9ada-834ba2866f41 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.159347 Jul 24 01:09:30.902946 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 477/961] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:30 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:31.919177 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:31.919798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:31.926336 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:31.926810 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:31.927586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:31.928054 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:32.060598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:32.068770 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f7165672-c1f9-4b7b-b8cf-dbe1914d8360 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.153528 Jul 24 01:09:32.069336 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 485/962] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:31 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:33.085512 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:33.086235 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:33.088819 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:33.094592 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:33.095051 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:33.095825 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:33.096268 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:33.236923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:33.245179 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9ac6af11-59aa-45ae-99ae-6f5a2e1b039a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.163678 Jul 24 01:09:33.245741 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 478/963] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:33 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:34.262612 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:34.263312 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:34.269819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:34.270263 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:34.271063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:34.271495 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:34.401613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:34.409297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9fc485b4-5e24-40bf-b184-2dd59258e18a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.151184 Jul 24 01:09:34.410114 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 486/964] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:34 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:34.897216 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-cd57f9ae-9628-4b24-b881-4ef541cd15f6 req-e152d57d-a822-4f86-8d02-17a4d96ca354 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "863bc791-b8b0-4b49-bf36-6dd4ab8b039d", "name": "network-vif-plugged", "status": "completed", "tag": "2dace519-865d-4dd1-84e9-96672860e4be"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:34.906636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-cd57f9ae-9628-4b24-b881-4ef541cd15f6 req-e152d57d-a822-4f86-8d02-17a4d96ca354 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:34.907463 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-cd57f9ae-9628-4b24-b881-4ef541cd15f6 req-e152d57d-a822-4f86-8d02-17a4d96ca354 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:34.922541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-cd57f9ae-9628-4b24-b881-4ef541cd15f6 req-e152d57d-a822-4f86-8d02-17a4d96ca354 service nova] Creating event network-vif-plugged:2dace519-865d-4dd1-84e9-96672860e4be for instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d on hv-cinder-maste Jul 24 01:09:34.927574 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-cd57f9ae-9628-4b24-b881-4ef541cd15f6 req-e152d57d-a822-4f86-8d02-17a4d96ca354 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035442 Jul 24 01:09:34.928541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 479/965] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:35.425478 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:35.426127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:35.433113 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:35.433649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:35.434532 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:35.435075 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:35.591177 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:35.599633 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e68c4c9c-9579-4915-bff7-d334ddc480ad tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1266 microversion: 2.1 time: 0.178071 Jul 24 01:09:35.600856 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 487/966] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:35 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1266 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:36.618370 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:36.619922 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:36.626138 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:36.626582 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:36.627378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:36.627815 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:36.760888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:36.768444 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-430759b5-429b-41ee-9f0e-03111e1af5bf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1453 microversion: 2.1 time: 0.154756 Jul 24 01:09:36.769071 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 480/967] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:36 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1453 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:36.784606 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:36.785220 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:36.791725 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:36.792175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:36.792941 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:36.793381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:36.924448 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:36.932594 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b2fac683-1605-4570-b700-4370d76a731a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1453 microversion: 2.1 time: 0.151828 Jul 24 01:09:36.933155 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 488/968] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:36 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1453 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:38.891462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-f15dc7ed-6492-4d50-9726-0b6c0a1e073a req-4aa3ea47-4a6a-4901-96ca-24d9b8d3d97c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "863bc791-b8b0-4b49-bf36-6dd4ab8b039d", "tag": "2dace519-865d-4dd1-84e9-96672860e4be"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:38.900507 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-f15dc7ed-6492-4d50-9726-0b6c0a1e073a req-4aa3ea47-4a6a-4901-96ca-24d9b8d3d97c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:38.901162 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-f15dc7ed-6492-4d50-9726-0b6c0a1e073a req-4aa3ea47-4a6a-4901-96ca-24d9b8d3d97c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:38.912782 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-f15dc7ed-6492-4d50-9726-0b6c0a1e073a req-4aa3ea47-4a6a-4901-96ca-24d9b8d3d97c service nova] Creating event network-changed:2dace519-865d-4dd1-84e9-96672860e4be for instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d on hv-cinder-maste Jul 24 01:09:38.919213 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-f15dc7ed-6492-4d50-9726-0b6c0a1e073a req-4aa3ea47-4a6a-4901-96ca-24d9b8d3d97c service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031462 Jul 24 01:09:38.920052 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 481/969] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:40.028252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1798797220", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-319784191", "security_groups": [{"name": "tempest-secgroup-smoke-1980147693"}], "networks": [{"uuid": "51a0ac89-567c-49c5-9e0e-90ef982eaba7"}], "block_device_mapping_v2": [{"uuid": "8773dbbe-d0aa-4aa2-a702-09fd24b777d7", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:40.138473 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] validate_networks() for [('51a0ac89-567c-49c5-9e0e-90ef982eaba7', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:09:40.502392 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:09:40.502845 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:09:40.503327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:09:40.503740 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:09:40.504154 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:09:40.513065 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:40.513518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:40.514321 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:40.514760 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:40.531742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Getting quotas for project 6af7b896ae8f4e09957a342a4c0d3030. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:09:40.535726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Getting quotas for user b7a89576949f471f9cd92bbc6677175f and project 6af7b896ae8f4e09957a342a4c0d3030. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:09:40.592224 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:09:40.594301 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8773dbbe-d0aa-4aa2-a702-09fd24b777d7',volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:09:40.720450 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8835a9d1-585b-4145-b6cc-4a107c5a0a17 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.696112 Jul 24 01:09:40.721092 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 489/970] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:09:40 2022] POST /compute/v2.1/servers => generated 406 bytes in 697 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:09:40.731676 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6d3e073d-5ffb-4803-888e-077c2a5dec42 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:40.732392 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6d3e073d-5ffb-4803-888e-077c2a5dec42 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:40.787540 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6d3e073d-5ffb-4803-888e-077c2a5dec42 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:40.795172 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6d3e073d-5ffb-4803-888e-077c2a5dec42 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1088 microversion: 2.1 time: 0.067658 Jul 24 01:09:40.795760 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 482/971] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:40 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1088 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:09:41.812004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:41.812701 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:41.815360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=54,nova_api:INSERT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:41.821649 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:41.822986 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:41.823327 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:41.823761 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:41.890860 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:41.903548 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fdb3b2d2-e83e-4a0a-a463-7e08b4d78e6e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1204 microversion: 2.1 time: 0.095281 Jul 24 01:09:41.904127 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 490/972] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:41 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1204 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:42.922027 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:42.922937 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:42.931181 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:42.932130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:42.932953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:42.933385 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:43.076890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:43.087702 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-58595c7b-f992-43d4-9302-088c9c190d93 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.168586 Jul 24 01:09:43.088452 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 483/973] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:42 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:09:43.098745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-377e1abf-b908-4afc-a90a-01612935281e req-aa86c5c4-9593-45c8-9b67-0e2b3e7bf613 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:43.116211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-377e1abf-b908-4afc-a90a-01612935281e req-aa86c5c4-9593-45c8-9b67-0e2b3e7bf613 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:43.119739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-377e1abf-b908-4afc-a90a-01612935281e req-aa86c5c4-9593-45c8-9b67-0e2b3e7bf613 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:43.125078 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=78 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:43.135610 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-377e1abf-b908-4afc-a90a-01612935281e req-aa86c5c4-9593-45c8-9b67-0e2b3e7bf613 service nova] Creating event network-changed:cff1233a-7129-41d1-b7f6-a56a7166e345 for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce on hv-cinder-maste Jul 24 01:09:43.140567 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-377e1abf-b908-4afc-a90a-01612935281e req-aa86c5c4-9593-45c8-9b67-0e2b3e7bf613 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045456 Jul 24 01:09:43.141720 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 491/974] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:44.105732 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:44.106345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:44.108816 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:44.114799 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:44.115279 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:44.116045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:44.116476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:44.253403 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:44.260941 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-336cbe69-f221-46e2-8aa1-e47798ab66be tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.159730 Jul 24 01:09:44.261506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 484/975] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:44 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:45.278635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:45.279368 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:45.286361 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:45.286850 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:45.287627 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:45.288067 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:45.419297 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:45.426793 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c706eb07-f6d1-45b5-a583-6c0571187d1b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.153068 Jul 24 01:09:45.427361 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 492/976] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:45 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:46.444509 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:46.445125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:46.452448 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:46.452901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:46.453666 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:46.454107 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:46.456683 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=71 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:46.587931 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:46.595613 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e903f900-f523-4b85-9784-42c81fe3f4f7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.155236 Jul 24 01:09:46.596182 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 485/977] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:46 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:47.612723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:47.613359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:47.619983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:47.620428 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:47.621207 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:47.621625 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:47.749899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:47.757342 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5cb02966-91cb-4d15-b0f9-3812c5cf30eb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.148572 Jul 24 01:09:47.757894 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 493/978] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:47 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:48.774971 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:48.776157 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:48.782577 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:48.783058 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:48.784218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:48.784357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:48.921876 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:48.929555 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ac121689-a40b-4f0b-bfdd-d913793dc77f tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.158547 Jul 24 01:09:48.930125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 486/979] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:48 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:49.946672 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:49.947615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:49.954222 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:49.954666 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:49.955503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:49.956032 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:50.083348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:50.091245 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8a300471-e252-40cf-b895-d08e4537e9c3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.148396 Jul 24 01:09:50.091815 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 494/980] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:49 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:51.107606 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:51.108214 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:51.110498 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:51.116323 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:51.116774 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:51.117545 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:51.118007 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:51.248218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:51.255628 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f1993b2e-1ad7-4ea6-9564-588aa73bd923 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.152562 Jul 24 01:09:51.256198 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 487/981] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:51 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:52.272248 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:52.272859 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:52.274872 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:52.275665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:09:52.282262 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:52.282819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:52.283680 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:52.284116 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:52.417185 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:52.425036 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b1195443-9647-4c23-ad79-c1bc57031b3b tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.156686 Jul 24 01:09:52.425604 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 495/982] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:52 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:52.967797 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-5f05525a-a0c1-49dc-9fed-70254340bccc req-13cc0226-9ed1-4613-b808-672c2568c9eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "name": "network-vif-plugged", "status": "completed", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:52.977547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-5f05525a-a0c1-49dc-9fed-70254340bccc req-13cc0226-9ed1-4613-b808-672c2568c9eb service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:52.978032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-5f05525a-a0c1-49dc-9fed-70254340bccc req-13cc0226-9ed1-4613-b808-672c2568c9eb service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:52.990000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-5f05525a-a0c1-49dc-9fed-70254340bccc req-13cc0226-9ed1-4613-b808-672c2568c9eb service nova] Creating event network-vif-plugged:cff1233a-7129-41d1-b7f6-a56a7166e345 for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce on hv-cinder-maste Jul 24 01:09:52.995875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-5f05525a-a0c1-49dc-9fed-70254340bccc req-13cc0226-9ed1-4613-b808-672c2568c9eb service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031845 Jul 24 01:09:52.996462 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 488/983] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:09:53.441469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:53.442379 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:53.449043 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:53.449491 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:53.450252 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:53.450685 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:53.578264 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:53.585715 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a226b88c-0d63-466b-8aa6-82c4bbdd248e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1260 microversion: 2.1 time: 0.148154 Jul 24 01:09:53.586265 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 496/984] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:53 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1260 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:54.602394 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:54.603177 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:54.609518 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:54.609976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:54.610745 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:54.611221 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:54.752759 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:54.761001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-43057e74-100a-468d-ae8f-9b0f428a2c44 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1443 microversion: 2.1 time: 0.163323 Jul 24 01:09:54.761568 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 489/985] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:54 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1443 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:54.776948 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:09:54.777564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:09:54.784057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:54.784505 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:54.785268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:54.785703 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:54.917284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:09:54.924829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9120edde-1f76-40d3-8e88-9ac7536dd9d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1443 microversion: 2.1 time: 0.151894 Jul 24 01:09:54.925393 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 497/986] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:09:54 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1443 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:09:56.586634 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-153e6fdf-0f6b-430c-8f12-99a6a32a70a8 req-aac64aaa-2848-446a-a21e-871283da1077 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:09:56.595043 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-153e6fdf-0f6b-430c-8f12-99a6a32a70a8 req-aac64aaa-2848-446a-a21e-871283da1077 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:09:56.595492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-153e6fdf-0f6b-430c-8f12-99a6a32a70a8 req-aac64aaa-2848-446a-a21e-871283da1077 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:09:56.607395 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-153e6fdf-0f6b-430c-8f12-99a6a32a70a8 req-aac64aaa-2848-446a-a21e-871283da1077 service nova] Creating event network-changed:cff1233a-7129-41d1-b7f6-a56a7166e345 for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce on hv-cinder-maste Jul 24 01:09:56.612413 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-153e6fdf-0f6b-430c-8f12-99a6a32a70a8 req-aac64aaa-2848-446a-a21e-871283da1077 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029458 Jul 24 01:09:56.613250 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 490/987] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:09:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:10:59.445219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:10:59.445770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:10:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:11:01.005339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:11:01.005887 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Sun Jul 24 01:11:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:11:35.919598 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=15 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:35.920377 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=37 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:35.924978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:35.925474 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:35.932288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:35.932745 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:35.933511 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:35.933948 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:35.960523 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:11:36.037927 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-6688f3e1-53d6-4382-8cc3-5157ef56fc85 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "DELETE /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 204 len: 0 microversion: 2.1 time: 0.119908 Jul 24 01:11:36.038503 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 498/988] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:11:35 2022] DELETE /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:11:36.045339 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=13 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:36.046113 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=27 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:36.051334 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:36.052071 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:36.059092 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:36.059542 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:36.060306 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:36.060743 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:36.304174 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:36.311796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f5359d3f-017d-41b6-9a78-a86cf85ccd53 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1578 microversion: 2.1 time: 0.268894 Jul 24 01:11:36.313453 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 491/989] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:36 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1578 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:37.330721 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:37.331357 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:37.337847 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:37.338296 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:37.339086 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:37.339524 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:37.519250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:37.526723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-102c5226-77d5-4d49-b09c-6b70cb4c2711 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1578 microversion: 2.1 time: 0.200936 Jul 24 01:11:37.527374 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 499/990] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:37 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1578 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:38.544719 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:38.545467 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:38.554914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:38.555134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:38.555880 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:38.556314 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:38.623514 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:38.630362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-fd419014-a91e-4d34-8da2-6345e24687ea req-e00f9852-04c6-4e30-b91c-df9d9d8bdb8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "863bc791-b8b0-4b49-bf36-6dd4ab8b039d", "name": "network-vif-deleted", "tag": "2dace519-865d-4dd1-84e9-96672860e4be"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:11:38.631772 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-f98982f6-73d7-4972-8df9-afc08c91a31a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 200 len: 1510 microversion: 2.1 time: 0.091234 Jul 24 01:11:38.632354 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 500/991] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:38 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 1510 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:11:38.640192 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-fd419014-a91e-4d34-8da2-6345e24687ea req-e00f9852-04c6-4e30-b91c-df9d9d8bdb8a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:38.641602 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-fd419014-a91e-4d34-8da2-6345e24687ea req-e00f9852-04c6-4e30-b91c-df9d9d8bdb8a service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:38.665111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-fd419014-a91e-4d34-8da2-6345e24687ea req-e00f9852-04c6-4e30-b91c-df9d9d8bdb8a service nova] Creating event network-vif-deleted:2dace519-865d-4dd1-84e9-96672860e4be for instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d on hv-cinder-maste Jul 24 01:11:38.670509 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-fd419014-a91e-4d34-8da2-6345e24687ea req-e00f9852-04c6-4e30-b91c-df9d9d8bdb8a service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.135206 Jul 24 01:11:38.671598 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 492/992] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:11:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:11:39.651427 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:39.652052 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:39.659788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:39.660242 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:39.661024 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:39.661457 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:39.676859 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] HTTP exception thrown: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Request method failure captured: Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d HTTP/1.1 Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jue5UKCAbGiAW1r_RNzRV6ogCQZtRChLqeUbe9Cpa8bYw2kmotTJdDbgbfZJcnFoD9f_MC_J7yN0_O1_NGx1dFQjp3QcqPxfaDCOFP-F9EIyXaoStD7sCDz_9W03UDH4HRNiaf2ZcSmc36yPN2YGFVnWWZlqhpC5VzJp5YT8M2FnWg Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:11:39.678550 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_be64d6ea0f0149608c9a823425a0e811", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-TestVolumeBootPattern-634892198 Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 7c545f7be7794566b639c29c0717c17a Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:11:39.680302 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '863bc791-b8b0-4b49-bf36-6dd4ab8b039d'} Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:11:39.681213 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:11:39.682531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:11:39.683737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Returning 404 to user: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:11:39.683737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1ba57baa-0e98-492f-b728-2644af831d6f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 404 len: 111 microversion: 2.1 time: 0.032474 Jul 24 01:11:39.683737 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 501/993] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:39 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:11:39.778247 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-212182022", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1886212345", "security_groups": [{"name": "tempest-secgroup-smoke-1806918878"}], "block_device_mapping_v2": [{"uuid": "9be425e4-32f5-49d6-91be-7bd6dbc5de73", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1b1fc3ab-4643-4369-871d-3a6b06dcc201"}]}} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:11:39.893875 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.network.neutron [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] validate_networks() for [('1b1fc3ab-4643-4369-871d-3a6b06dcc201', None, None, None, None, None)] {{(pid=117955) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:11:40.262285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Flavor limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:11:40.262712 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Image limits 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:11:40.263223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Flavor pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:11:40.263635 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Image pref 0:0:0 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:11:40.264050 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.virt.hardware [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117955) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:11:40.272939 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:40.273391 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:40.274197 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:40.274639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:40.291294 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for project be64d6ea0f0149608c9a823425a0e811. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:11:40.295378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.quota [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for user 7c545f7be7794566b639c29c0717c17a and project be64d6ea0f0149608c9a823425a0e811. Resources: {'cores', 'instances', 'ram'} {{(pid=117955) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:11:40.352289 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Going to run 1 instances... {{(pid=117955) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:11:40.354467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='9be425e4-32f5-49d6-91be-7bd6dbc5de73',volume_size=None,volume_type=None)] {{(pid=117955) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:11:40.481488 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8f2ded8e-f01d-4cb2-812d-4b8a9b926ae4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.707370 Jul 24 01:11:40.487537 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 493/994] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:11:39 2022] POST /compute/v2.1/servers => generated 406 bytes in 709 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:11:40.493013 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-66651b1b-8ac7-4f55-8da1-3e45db860d73 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:40.493652 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-66651b1b-8ac7-4f55-8da1-3e45db860d73 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:40.545312 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-66651b1b-8ac7-4f55-8da1-3e45db860d73 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:40.552538 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-66651b1b-8ac7-4f55-8da1-3e45db860d73 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1094 microversion: 2.1 time: 0.063474 Jul 24 01:11:40.553102 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 502/995] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:40 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1094 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:41.570480 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:41.571125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:41.578435 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:41.578925 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:41.579705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:41.580351 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:41.643617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:41.651323 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5bcccfd6-33cc-4b5d-abb9-49a133f09065 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1200 microversion: 2.1 time: 0.085516 Jul 24 01:11:41.651894 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 494/996] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:41 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1200 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:42.668628 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:42.669258 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:42.679759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:42.680209 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:42.680985 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:42.681713 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:42.826105 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:42.834120 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1a727f8b-19c2-4260-a3c9-2b8f808d7c0c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.169240 Jul 24 01:11:42.835423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 503/997] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:42 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:42.987184 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-979d4072-c624-476a-8657-1efda249993a req-58eb36fe-0268-44e7-9a13-7884a53de07d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "495ea0b1-09f6-45d1-9dc1-f47bd463db92", "tag": "a808a619-ba7a-4d1d-946b-70726603a48f"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:11:42.996275 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-979d4072-c624-476a-8657-1efda249993a req-58eb36fe-0268-44e7-9a13-7884a53de07d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:42.996995 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-979d4072-c624-476a-8657-1efda249993a req-58eb36fe-0268-44e7-9a13-7884a53de07d service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:43.009495 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-979d4072-c624-476a-8657-1efda249993a req-58eb36fe-0268-44e7-9a13-7884a53de07d service nova] Creating event network-changed:a808a619-ba7a-4d1d-946b-70726603a48f for instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 on hv-cinder-maste Jul 24 01:11:43.022066 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-979d4072-c624-476a-8657-1efda249993a req-58eb36fe-0268-44e7-9a13-7884a53de07d service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049032 Jul 24 01:11:43.022685 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 495/998] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:11:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:11:43.852029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:43.852671 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:43.859596 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:43.859771 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:43.860682 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:43.861436 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:43.994183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:44.001960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-31f5f474-a1e5-45d4-a9aa-1a3fcf03521b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.153860 Jul 24 01:11:44.002536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 504/999] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:43 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:45.019831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:45.020522 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:45.026703 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:45.027181 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:45.027937 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:45.028372 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:45.157648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:45.165122 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9831f938-8bcc-48b0-94f2-2a054e68cac7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.150125 Jul 24 01:11:45.165684 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 496/1000] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:45 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:46.181420 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:46.182026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:46.189232 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:46.189696 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:46.190431 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:46.190887 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:46.323008 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:46.330446 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8d4b2d72-7686-4f93-8420-b9004e88d7fe tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.152951 Jul 24 01:11:46.331039 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 505/1001] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:46 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:47.347008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:47.347637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:47.354113 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:47.354569 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:47.355378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:47.355821 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:47.488180 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:47.495667 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1efd1065-2f5d-4dde-8590-a290627bfeaf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.152578 Jul 24 01:11:47.496236 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 497/1002] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:47 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:48.513011 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:48.513624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:48.520712 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:48.521166 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:48.521942 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:48.522381 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:48.654962 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:48.662708 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-d3799a51-f250-45ab-a447-1ca3b26557a4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.153586 Jul 24 01:11:48.663307 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 506/1003] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:48 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:49.679807 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:49.680411 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:49.687045 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:49.687492 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:49.688249 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:49.688792 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:49.817285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:49.825004 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a16d5871-f2d0-4ec6-b012-6f937e7d3648 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.149042 Jul 24 01:11:49.825573 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 498/1004] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:49 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:11:50.841394 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:50.842004 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:50.845294 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:50.850738 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:50.851219 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:50.851984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:50.852422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:50.985020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:50.993159 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-002f827b-69a8-4525-8518-c224f4129041 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.155664 Jul 24 01:11:50.993726 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 507/1005] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:50 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:52.009501 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:52.010132 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:52.012060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:52.012717 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=6 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:52.019739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:52.020225 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:52.020997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:52.021427 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:52.152855 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:52.160468 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-77a2c994-6c89-4037-83b4-a553e0ea5149 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.154885 Jul 24 01:11:52.161037 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 499/1006] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:52 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:53.177340 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:53.177956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:53.184553 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:53.185006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:53.185777 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:53.186218 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:53.316259 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:53.323984 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-a95d81e5-e77d-4a24-98e5-4aa619afe515 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.150657 Jul 24 01:11:53.324543 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 508/1007] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:53 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:53.619019 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-cca84120-6fd1-4331-b2f2-a493feced641 req-9fa51877-eebb-4f4a-973b-ecbc084468bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "495ea0b1-09f6-45d1-9dc1-f47bd463db92", "name": "network-vif-plugged", "status": "completed", "tag": "a808a619-ba7a-4d1d-946b-70726603a48f"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:11:53.629259 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-cca84120-6fd1-4331-b2f2-a493feced641 req-9fa51877-eebb-4f4a-973b-ecbc084468bc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:53.629706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-cca84120-6fd1-4331-b2f2-a493feced641 req-9fa51877-eebb-4f4a-973b-ecbc084468bc service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:53.640617 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-cca84120-6fd1-4331-b2f2-a493feced641 req-9fa51877-eebb-4f4a-973b-ecbc084468bc service nova] Creating event network-vif-plugged:a808a619-ba7a-4d1d-946b-70726603a48f for instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 on hv-cinder-maste Jul 24 01:11:53.645506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-cca84120-6fd1-4331-b2f2-a493feced641 req-9fa51877-eebb-4f4a-973b-ecbc084468bc service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087979 Jul 24 01:11:53.646093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 500/1008] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:11:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:11:53.926019 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:11:53.926565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:53.932947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:53.933393 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:53.934172 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:53.934603 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:53.958696 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.objects.instance [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lazy-loading 'flavor' on Instance uuid dce7c08a-cea9-4fe0-a0e5-5f6224f293ce {{(pid=117956) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:11:54.036362 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ee0daca5-7563-4e5c-a660-df07ccf2851d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "POST /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce/action" status: 202 len: 0 microversion: 2.1 time: 0.115108 Jul 24 01:11:54.037026 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 509/1009] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:11:53 2022] POST /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce/action => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:11:54.047319 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:54.049510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:54.056158 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:54.056647 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:54.057659 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:54.058093 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:54.200654 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:54.208223 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a7baa977-0c69-4f14-81dd-b6774c5fb0f9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.166464 Jul 24 01:11:54.208790 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 501/1010] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:54 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:54.341836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:54.342565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:54.349495 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:54.349959 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:54.350780 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:54.351230 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:54.479125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:54.486788 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-1271db57-6452-43be-a5f9-85bb2f334813 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1266 microversion: 2.1 time: 0.149678 Jul 24 01:11:54.487359 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 510/1011] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:54 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:55.225485 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:55.226160 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:55.232780 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:55.233234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:55.233999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:55.234620 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:55.370042 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:55.377709 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-96978ffc-21bc-4dee-b46c-dfcb5e3d37ad tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.156162 Jul 24 01:11:55.378276 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 502/1012] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:55 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:55.502943 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:55.503558 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:55.510013 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:55.510460 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:55.511260 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:55.511698 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:55.644723 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:55.652603 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-5c338e5b-036f-4617-80b2-f2ebd4450f8a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1454 microversion: 2.1 time: 0.153558 Jul 24 01:11:55.653183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 511/1013] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:55 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1454 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:55.670205 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:55.670996 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:55.677667 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:55.678201 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:55.679001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:55.679443 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:55.811494 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:55.819762 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a607798e-43da-43bc-b6c1-8ec3a36e8740 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1454 microversion: 2.1 time: 0.153399 Jul 24 01:11:55.820332 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 503/1014] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:55 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1454 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:56.394798 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:56.395413 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:56.401508 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:56.401962 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:56.402720 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:56.403183 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:56.423285 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:11:56.597017 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:56.604769 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-09818983-6b24-4a17-9877-cf2096cfd603 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.213914 Jul 24 01:11:56.605275 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 512/1015] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:56 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:57.621421 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:57.622038 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:57.628547 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:57.628999 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:57.629775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:57.630212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:57.817742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:57.826804 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-4103546a-1a0e-4b5d-b2a0-80db4ba9a652 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.209246 Jul 24 01:11:57.827554 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 504/1016] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:57 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:11:57.858957 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-924067ac-8526-42f6-912b-4bc36bc12e77 req-610d79b8-fe40-49aa-b63d-b70feee58994 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "495ea0b1-09f6-45d1-9dc1-f47bd463db92", "tag": "a808a619-ba7a-4d1d-946b-70726603a48f"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:11:57.867898 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-924067ac-8526-42f6-912b-4bc36bc12e77 req-610d79b8-fe40-49aa-b63d-b70feee58994 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:57.868422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-924067ac-8526-42f6-912b-4bc36bc12e77 req-610d79b8-fe40-49aa-b63d-b70feee58994 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:57.879716 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-924067ac-8526-42f6-912b-4bc36bc12e77 req-610d79b8-fe40-49aa-b63d-b70feee58994 service nova] Creating event network-changed:a808a619-ba7a-4d1d-946b-70726603a48f for instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 on hv-cinder-maste Jul 24 01:11:57.884921 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-924067ac-8526-42f6-912b-4bc36bc12e77 req-610d79b8-fe40-49aa-b63d-b70feee58994 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094966 Jul 24 01:11:57.885530 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 513/1017] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:11:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:11:58.844460 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:11:58.845563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:11:58.852090 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:58.852538 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:58.853296 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:11:58.853725 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:11:58.985788 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:11:58.993409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-758a29d3-a8a6-4cd8-b481-c3a319d298ef tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.153240 Jul 24 01:11:58.994008 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 505/1018] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:11:58 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:00.012228 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:00.012836 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:00.019728 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:00.020175 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:00.020935 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:00.021371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:00.148267 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:00.155804 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3e451025-f2fc-4ac3-994d-01a2c57224c2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.148426 Jul 24 01:12:00.156372 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 514/1019] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:00 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:01.173787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:01.174428 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:01.180663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:01.181111 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:01.181860 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:01.182290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:01.312310 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:01.321040 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-b69dcf29-a310-4a95-80b5-0a540cc856b9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.151814 Jul 24 01:12:01.321638 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 506/1020] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:01 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:02.337487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:02.338101 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:02.345235 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:02.345716 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:02.346488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:02.346978 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:02.481440 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:02.488933 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-9708bdfe-b812-4a49-bef0-d5aedb9c69bb tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.155432 Jul 24 01:12:02.489487 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 515/1021] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:02 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:03.505415 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:03.506027 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:03.512637 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:03.513113 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:03.513873 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:03.514358 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:03.643219 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:03.650817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3a92ccb1-e8b0-48b5-bed8-a02782f9d4f5 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.149287 Jul 24 01:12:03.651389 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 507/1022] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:03 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:04.667695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:04.668422 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:04.676789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:04.677241 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:04.678014 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:04.678449 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:04.810751 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:04.819278 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-e4a2f33e-417e-4a9f-985f-489a23e2dcc9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.155400 Jul 24 01:12:04.819810 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 516/1023] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:04 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:05.836624 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:05.837212 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:05.843760 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:05.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:05.844991 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:05.845938 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:05.974755 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:05.982299 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-78aced1a-c3b9-4019-b0f2-3e08536a46c6 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.149720 Jul 24 01:12:05.982883 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 508/1024] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:05 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:06.999229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:06.999828 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:07.002407 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:12:07.007892 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:07.008288 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:07.009081 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:07.009562 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:07.140481 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:07.148636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-20625884-d009-4a7d-9538-3bed065b0e6d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1569 microversion: 2.1 time: 0.153274 Jul 24 01:12:07.149216 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 517/1025] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:06 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1569 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:07.165306 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:12:07.165909 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:07.168377 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=50,nova_api:INSERT=3 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:12:07.173611 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:07.174068 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:07.174864 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:07.175308 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:07.179690 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=93 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:12:07.266136 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-42b809a8-74be-44ce-96b5-6fd90ddbad25 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "POST /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce/action" status: 202 len: 0 microversion: 2.1 time: 0.104584 Jul 24 01:12:07.267848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 509/1026] 10.222.0.18 () {60 vars in 1277 bytes} [Sun Jul 24 01:12:07 2022] POST /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce/action => generated 0 bytes in 106 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 24 01:12:07.278972 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:07.279872 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:07.287636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:07.288083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:07.289169 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:07.289604 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:07.430357 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:07.437858 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-b9dfc791-2a61-40d2-84b1-d79294a08128 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1517 microversion: 2.1 time: 0.162868 Jul 24 01:12:07.438423 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 518/1027] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:07 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1517 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:08.456736 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:08.457353 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:08.463663 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:08.464125 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:08.464903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:08.465365 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:08.599825 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:08.607795 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-37c3af8e-afa7-41cf-817c-fd8bc8b3b26c tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1517 microversion: 2.1 time: 0.156048 Jul 24 01:12:08.608450 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 510/1028] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:08 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1517 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:09.619344 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-32d356d2-6d97-4185-9909-663b9b90a096 req-36fd8bd6-4d18-4128-bc3e-8e451d2f2ad1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "name": "network-vif-unplugged", "status": "completed", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:12:09.626054 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:09.626994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:09.629749 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-32d356d2-6d97-4185-9909-663b9b90a096 req-36fd8bd6-4d18-4128-bc3e-8e451d2f2ad1 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:09.629904 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-32d356d2-6d97-4185-9909-663b9b90a096 req-36fd8bd6-4d18-4128-bc3e-8e451d2f2ad1 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:09.633688 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:09.634139 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:09.635131 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:09.635578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:09.641615 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.compute.server_external_events [req-32d356d2-6d97-4185-9909-663b9b90a096 req-36fd8bd6-4d18-4128-bc3e-8e451d2f2ad1 service nova] Unable to find a host for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce. Dropping event network-vif-unplugged {{(pid=117956) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 24 01:12:09.642827 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-32d356d2-6d97-4185-9909-663b9b90a096 req-36fd8bd6-4d18-4128-bc3e-8e451d2f2ad1 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.027032 Jul 24 01:12:09.643746 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 519/1029] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:12:09 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 29 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 24 01:12:09.771648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:09.779253 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-68baef18-c986-4dfe-bbf2-a9790dab0560 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1517 microversion: 2.1 time: 0.157507 Jul 24 01:12:09.779831 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 511/1030] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:09 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1517 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:10.795969 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:10.796583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:10.803518 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:10.803961 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:10.804714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:10.805144 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:10.937909 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:10.946695 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7ef2cf59-b582-40f9-9cf8-16d283d58e3e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1517 microversion: 2.1 time: 0.154653 Jul 24 01:12:10.947284 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 520/1031] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:10 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1517 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:11.964577 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:11.965285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:11.971890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:11.972338 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:11.973119 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:11.973552 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:12.107997 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:12.115868 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5d04f11d-c528-4a28-88d6-2b55404e3994 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1517 microversion: 2.1 time: 0.155704 Jul 24 01:12:12.116430 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 512/1032] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:11 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1517 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:13.133354 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:13.133924 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:13.140323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:13.140768 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:13.141520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:13.141947 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:13.273269 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:13.280899 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-54e33e7c-cafc-4d55-adb0-3cfbc261561e tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.151506 Jul 24 01:12:13.281468 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 521/1033] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:13 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:14.299817 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:14.300923 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:14.308105 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:14.308554 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:14.309331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:14.309763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:14.457331 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:14.465480 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f7d77333-349e-470a-99c3-ec8d78a0cb48 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.170355 Jul 24 01:12:14.466053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 513/1034] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:14 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:15.484279 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:15.484888 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:15.492150 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:15.492595 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:15.493366 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:15.493801 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:15.626983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:15.634444 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-8af38dd0-deef-4f05-8e81-435153a18dc7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.154095 Jul 24 01:12:15.635057 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 522/1035] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:15 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:16.651322 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:16.651932 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:16.658727 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:16.659254 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:16.660013 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:16.660447 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:16.786977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:16.794499 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-a744fe3b-9173-4480-a0cb-f8f0c6828e9d tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.147102 Jul 24 01:12:16.795087 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 514/1036] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:16 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:17.812000 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:17.812544 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:17.819953 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:17.820398 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:17.821157 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:17.821593 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:17.952076 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:17.960200 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-3a247d6f-baba-4536-b363-6d2ce9e5cfb1 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.152202 Jul 24 01:12:17.960776 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 523/1037] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:17 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:18.977577 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:18.978175 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:18.984700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:18.985148 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:18.985903 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:18.986671 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:19.113301 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:19.120742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-1ea8c909-13e6-4b4e-9c42-b75c03aeef32 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.147031 Jul 24 01:12:19.121290 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 515/1038] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:18 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:20.137472 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:20.138083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:20.144837 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:20.145313 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:20.146087 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:20.146526 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:20.273618 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:20.281152 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-cb09a3b5-02b9-42e9-859d-34c378515fb4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.147651 Jul 24 01:12:20.281724 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 524/1039] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:20 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:21.299735 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:21.300383 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:21.306978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:21.307433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:21.308202 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:21.308644 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:21.441557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:21.449291 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-293dd7e2-f79f-4022-af6e-866644ab28c7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.154460 Jul 24 01:12:21.449848 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 516/1040] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:21 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:22.466141 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:22.466750 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:22.473263 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:22.473714 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:22.474480 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:22.474949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:22.602855 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:22.610348 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-c9cf3625-a8ff-4d3c-bef5-308df73ff2d7 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.148160 Jul 24 01:12:22.610944 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 525/1041] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:22 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:23.627467 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:23.628215 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:23.634739 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:23.635242 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:23.636019 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:23.636455 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:23.741835 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-bcd86b7d-55a2-4ce1-933b-9273367bb2f4 req-1094b5b8-5117-470a-a236-fb8c7401db69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "name": "network-vif-plugged", "status": "completed", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:12:23.753142 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-bcd86b7d-55a2-4ce1-933b-9273367bb2f4 req-1094b5b8-5117-470a-a236-fb8c7401db69 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:23.753739 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-bcd86b7d-55a2-4ce1-933b-9273367bb2f4 req-1094b5b8-5117-470a-a236-fb8c7401db69 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:23.768878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-bcd86b7d-55a2-4ce1-933b-9273367bb2f4 req-1094b5b8-5117-470a-a236-fb8c7401db69 service nova] Creating event network-vif-plugged:cff1233a-7129-41d1-b7f6-a56a7166e345 for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce on hv-cinder-maste Jul 24 01:12:23.774430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-bcd86b7d-55a2-4ce1-933b-9273367bb2f4 req-1094b5b8-5117-470a-a236-fb8c7401db69 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036644 Jul 24 01:12:23.774872 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:23.775410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 526/1042] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:12:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:12:23.782378 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7ee73f60-08ca-4e69-bfdb-49dadd6e4ee9 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.158803 Jul 24 01:12:23.783870 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 517/1043] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:23 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:24.672034 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:12:24.672645 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:12:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:12:24.801064 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:24.801757 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:24.808015 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:24.808465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:24.809229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:24.809665 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:24.938573 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:24.946635 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-7161f18d-3720-4d35-9b9c-8f7fe029ffa4 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1575 microversion: 2.1 time: 0.151190 Jul 24 01:12:24.947229 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 527/1044] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:24 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1575 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:25.964407 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:12:25.965009 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:12:25.972264 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:25.972721 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:25.973493 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:12:25.973933 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:12:26.106648 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:12:26.114914 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0b3a7e95-3a49-4664-8352-99255d467396 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1562 microversion: 2.1 time: 0.154372 Jul 24 01:12:26.115486 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 518/1045] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:12:25 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1562 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:12:26.275488 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:12:26.275976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Sun Jul 24 01:12:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:13:42.392716 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=21 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:42.394761 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=21 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:42.395751 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=80 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:42.395881 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:42.401106 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:42.401476 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [req-1245e543-76f8-47cb-96c1-0b5b748ab417 req-803819e4-f360-4961-9fd6-8f2c291afa41 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:13:42.401830 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:42.410549 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:42.411366 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:42.412624 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:42.413268 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:42.414717 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-1245e543-76f8-47cb-96c1-0b5b748ab417 req-803819e4-f360-4961-9fd6-8f2c291afa41 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:42.415384 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [req-1245e543-76f8-47cb-96c1-0b5b748ab417 req-803819e4-f360-4961-9fd6-8f2c291afa41 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:42.431060 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.compute.server_external_events [req-1245e543-76f8-47cb-96c1-0b5b748ab417 req-803819e4-f360-4961-9fd6-8f2c291afa41 service nova] Creating event network-changed:cff1233a-7129-41d1-b7f6-a56a7166e345 for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce on hv-cinder-maste Jul 24 01:13:42.436953 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [req-1245e543-76f8-47cb-96c1-0b5b748ab417 req-803819e4-f360-4961-9fd6-8f2c291afa41 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044176 Jul 24 01:13:42.437782 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 519/1046] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:13:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:13:42.443182 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Going to try to terminate instance {{(pid=117956) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:13:42.541565 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-fca9a586-78ef-45d1-b83b-f86a5d56e3e2 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "DELETE /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 204 len: 0 microversion: 2.1 time: 0.150787 Jul 24 01:13:42.542340 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 528/1047] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:13:42 2022] DELETE /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 0 bytes in 153 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:13:42.552000 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:42.552829 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:42.560541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:42.561128 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:42.562147 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:42.562667 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:42.699056 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:42.706879 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6b5d8b5a-d3b0-4878-99ad-503b59f3ece3 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.159652 Jul 24 01:13:42.707781 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 520/1048] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:42 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:43.623063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-144878515", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1886212345", "security_groups": [{"name": "tempest-secgroup-smoke-1806918878"}], "block_device_mapping_v2": [{"uuid": "f52ee341-6cd9-469e-82b5-c7dd0ea266a6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1b1fc3ab-4643-4369-871d-3a6b06dcc201"}]}} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:13:43.725856 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:43.726475 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:43.732842 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:43.733597 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:43.734603 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:43.735296 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:43.737430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.network.neutron [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] validate_networks() for [('1b1fc3ab-4643-4369-871d-3a6b06dcc201', None, None, None, None, None)] {{(pid=117956) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 24 01:13:43.943856 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:43.954742 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-9e1bce0f-5458-46fc-8e97-6b57e15ee53a tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1568 microversion: 2.1 time: 0.233756 Jul 24 01:13:43.955890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 521/1049] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:43 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1568 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:44.230125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Flavor limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 24 01:13:44.230616 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Image limits 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 24 01:13:44.231128 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Flavor pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 24 01:13:44.231538 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Image pref 0:0:0 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 24 01:13:44.231950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.virt.hardware [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117956) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 24 01:13:44.240831 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:44.241387 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:44.242082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:44.242925 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:44.261466 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for project be64d6ea0f0149608c9a823425a0e811. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 24 01:13:44.265585 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.quota [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Getting quotas for user 7c545f7be7794566b639c29c0717c17a and project be64d6ea0f0149608c9a823425a0e811. Resources: {'cores', 'instances', 'ram'} {{(pid=117956) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 24 01:13:44.332323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Going to run 1 instances... {{(pid=117956) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 24 01:13:44.334743 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='f52ee341-6cd9-469e-82b5-c7dd0ea266a6',volume_size=None,volume_type=None)] {{(pid=117956) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 24 01:13:44.486079 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-608f9551-af68-4978-aec0-68a1fbb8ad3c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.867180 Jul 24 01:13:44.486919 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 529/1050] 10.222.0.18 () {60 vars in 1146 bytes} [Sun Jul 24 01:13:43 2022] POST /compute/v2.1/servers => generated 406 bytes in 869 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 24 01:13:44.497318 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7c04477b-8f7b-4f5d-b882-2368a6ce9f4f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:44.498234 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7c04477b-8f7b-4f5d-b882-2368a6ce9f4f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:44.577348 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-7c04477b-8f7b-4f5d-b882-2368a6ce9f4f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:44.585737 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7c04477b-8f7b-4f5d-b882-2368a6ce9f4f tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1094 microversion: 2.1 time: 0.092315 Jul 24 01:13:44.586318 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 522/1051] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:44 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1094 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:44.767608 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-f830e92b-835e-4fe4-b3d1-ea554f3d5e30 req-433dff6e-4c98-4ec1-9438-8520829899fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dce7c08a-cea9-4fe0-a0e5-5f6224f293ce", "name": "network-vif-deleted", "tag": "cff1233a-7129-41d1-b7f6-a56a7166e345"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:13:44.777964 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-f830e92b-835e-4fe4-b3d1-ea554f3d5e30 req-433dff6e-4c98-4ec1-9438-8520829899fe service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:44.778408 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-f830e92b-835e-4fe4-b3d1-ea554f3d5e30 req-433dff6e-4c98-4ec1-9438-8520829899fe service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:44.790266 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-f830e92b-835e-4fe4-b3d1-ea554f3d5e30 req-433dff6e-4c98-4ec1-9438-8520829899fe service nova] Creating event network-vif-deleted:cff1233a-7129-41d1-b7f6-a56a7166e345 for instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce on hv-cinder-maste Jul 24 01:13:44.796589 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-f830e92b-835e-4fe4-b3d1-ea554f3d5e30 req-433dff6e-4c98-4ec1-9438-8520829899fe service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033343 Jul 24 01:13:44.797303 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 530/1052] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:13:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:13:44.972803 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:44.973565 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:44.980114 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:44.980563 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:44.981355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:44.981787 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:45.051024 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a89576949f471f9cd92bbc6677175f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6af7b896ae8f4e09957a342a4c0d3030', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:45.060298 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-536d1d13-e494-44d1-8a12-f1dd7fe33527 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 200 len: 1219 microversion: 2.1 time: 0.091184 Jul 24 01:13:45.062579 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 523/1053] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:44 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:13:45.603732 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:45.604440 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:45.610878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:45.611323 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:45.612110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:45.612539 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:45.677975 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:45.685428 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ce3fd160-2403-440f-a4c1-0c08582fd541 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1210 microversion: 2.1 time: 0.086031 Jul 24 01:13:45.685981 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 531/1054] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:45 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:46.079595 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:46.080280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] [instance: dce7c08a-cea9-4fe0-a0e5-5f6224f293ce] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:46.088323 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:46.088770 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:46.089543 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:46.089977 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:46.104244 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] HTTP exception thrown: Instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce could not be found. Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Request method failure captured: Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce HTTP/1.1 Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Ju7dDd3GQhy2y3CNG_QX2iY_KH38aoYIbj4t1P88wXey6Yi6tvP65ElBfPOeORn4phAJFN2UUZXjA9rRaLTcnw5s6CN2CEjgGrSak0CIWlxJmmlTGbIQPQxedD5QmUVjXyVOEFje6TLpV0yhvSdDN3fSXHCVb7m-mfMNTugK4SQObs Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: 6af7b896ae8f4e09957a342a4c0d3030 Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-TestShelveInstance-1945545547 Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:13:46.105890 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/6af7b896ae8f4e09957a342a4c0d3030"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/6af7b896ae8f4e09957a342a4c0d3030"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/6af7b896ae8f4e09957a342a4c0d3030"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_6af7b896ae8f4e09957a342a4c0d3030", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-TestShelveInstance-1945545547 Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: 6af7b896ae8f4e09957a342a4c0d3030 Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-TestShelveInstance-1945545547 Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-TestShelveInstance-1945545547-project Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: b7a89576949f471f9cd92bbc6677175f Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-TestShelveInstance-1945545547-project Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce could not be found. Jul 24 01:13:46.107706 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': 'dce7c08a-cea9-4fe0-a0e5-5f6224f293ce'} Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce could not be found. Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:13:46.108902 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce could not be found. Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce could not be found. Jul 24 01:13:46.110269 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:13:46.111784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Returning 404 to user: Instance dce7c08a-cea9-4fe0-a0e5-5f6224f293ce could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:13:46.111784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-bb4c44dd-4808-41a5-82ef-121b39528489 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce" status: 404 len: 111 microversion: 2.1 time: 0.031851 Jul 24 01:13:46.111784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 524/1055] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:46 2022] GET /compute/v2.1/servers/dce7c08a-cea9-4fe0-a0e5-5f6224f293ce => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:13:46.283983 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-24b665f6-960c-4b0c-8fee-07d0cbdb5104 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] Calling method '.version_select at 0x7f5e305268b0>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:46.326155 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-24b665f6-960c-4b0c-8fee-07d0cbdb5104 tempest-TestShelveInstance-1945545547 tempest-TestShelveInstance-1945545547-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-319784191" status: 202 len: 0 microversion: 2.1 time: 0.045376 Jul 24 01:13:46.331425 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 532/1056] 10.222.0.18 () {58 vars in 1250 bytes} [Sun Jul 24 01:13:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-319784191 => generated 0 bytes in 48 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:13:46.703661 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:46.704307 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:46.712187 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:46.712691 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:46.713433 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:46.713878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:46.940349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:46.949075 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-ac48213f-81e9-482b-899e-01a456db6127 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.249315 Jul 24 01:13:46.958516 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 525/1057] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:46 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:47.406748 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-cb76d2a1-5b81-4906-a7a3-6fa7e9cae557 req-e5563a88-8093-4245-b50d-861e2bacd37f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "027a7e7d-2e80-453f-8b90-341fb9877ce8", "tag": "a628883e-1113-4579-bb10-1d2ff4bb2dc2"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:13:47.431779 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-cb76d2a1-5b81-4906-a7a3-6fa7e9cae557 req-e5563a88-8093-4245-b50d-861e2bacd37f service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:47.432360 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-cb76d2a1-5b81-4906-a7a3-6fa7e9cae557 req-e5563a88-8093-4245-b50d-861e2bacd37f service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:47.446243 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-cb76d2a1-5b81-4906-a7a3-6fa7e9cae557 req-e5563a88-8093-4245-b50d-861e2bacd37f service nova] Creating event network-changed:a628883e-1113-4579-bb10-1d2ff4bb2dc2 for instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 on hv-cinder-maste Jul 24 01:13:47.451556 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-cb76d2a1-5b81-4906-a7a3-6fa7e9cae557 req-e5563a88-8093-4245-b50d-861e2bacd37f service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058505 Jul 24 01:13:47.452250 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 533/1058] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:13:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:13:47.967345 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:47.968074 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:47.977022 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:47.977662 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:47.978843 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:47.979729 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:48.182285 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:48.193711 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-3aa0062e-0039-42ef-9662-b706a5200acd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.230531 Jul 24 01:13:48.194557 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 526/1059] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:47 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:49.212082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:49.212789 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:49.219762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:49.220211 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:49.220998 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:49.221852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:49.450992 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:49.459657 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-03446186-ee4e-433a-92a4-8b753fabe8f4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.252271 Jul 24 01:13:49.460530 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 534/1060] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:49 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:50.477484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:50.478130 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:50.484629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:50.485100 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:50.485858 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:50.486297 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:50.626718 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:50.636021 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-90452f81-ff6b-4355-9212-3e067ccad8a5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.162396 Jul 24 01:13:50.636610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 527/1061] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:50 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:51.711949 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:51.713419 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:51.720119 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:51.720914 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:51.721711 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:51.722199 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:51.869479 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:51.877013 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ec68d9ff-ce2f-4afb-8267-d7bdc6359a02 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.228862 Jul 24 01:13:51.877574 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 535/1062] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:51 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:52.894080 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:52.895026 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:52.902121 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:52.902764 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:52.912996 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:52.913506 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:53.057666 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:53.065342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-6b8e2198-e04b-4b1b-ab73-c7b2921e48d5 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.175134 Jul 24 01:13:53.066002 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 528/1063] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:52 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:54.083305 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:54.084054 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:54.090931 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:54.091387 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:54.092177 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:54.092613 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:54.223734 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:54.231179 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ec45110e-f66b-4e5f-b28c-405ba598c253 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.152414 Jul 24 01:13:54.231759 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 536/1064] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:54 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:55.247631 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:55.248241 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:55.250630 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell0:SELECT=2 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:55.256976 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:55.257454 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:55.258227 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:55.258689 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:55.390784 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:55.399031 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-d7756ce7-1905-41fe-ae8a-ee4584595a22 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.155169 Jul 24 01:13:55.399615 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 529/1065] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:55 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:56.415795 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:56.416430 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:56.418577 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=2 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:56.419391 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell0:SELECT=6 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:13:56.424985 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:56.426227 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:56.426878 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:56.427316 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:56.555134 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:56.562583 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-44e0df27-44d3-405c-beca-bc2e1e7a07cb tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.150689 Jul 24 01:13:56.563164 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 537/1066] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:56 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:57.580561 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:57.581169 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:57.588123 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:57.588600 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:57.589366 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:57.589802 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:57.724206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:57.731754 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-8a43063a-73ad-4bbe-a4fb-3b68eb82d2c8 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.155110 Jul 24 01:13:57.732318 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 530/1067] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:57 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:58.270829 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-31313db8-4c86-4b4c-9a89-3719eff4fdee req-adc076b0-55c9-483d-a336-7ee75014b139 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "027a7e7d-2e80-453f-8b90-341fb9877ce8", "name": "network-vif-plugged", "status": "completed", "tag": "a628883e-1113-4579-bb10-1d2ff4bb2dc2"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:13:58.279731 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-31313db8-4c86-4b4c-9a89-3719eff4fdee req-adc076b0-55c9-483d-a336-7ee75014b139 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:58.280074 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-31313db8-4c86-4b4c-9a89-3719eff4fdee req-adc076b0-55c9-483d-a336-7ee75014b139 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:58.291636 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-31313db8-4c86-4b4c-9a89-3719eff4fdee req-adc076b0-55c9-483d-a336-7ee75014b139 service nova] Creating event network-vif-plugged:a628883e-1113-4579-bb10-1d2ff4bb2dc2 for instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 on hv-cinder-maste Jul 24 01:13:58.296469 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-31313db8-4c86-4b4c-9a89-3719eff4fdee req-adc076b0-55c9-483d-a336-7ee75014b139 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030856 Jul 24 01:13:58.297046 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 538/1068] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:13:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:13:58.750674 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:58.751258 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:58.757891 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:58.758342 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:58.759143 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:58.759585 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:58.887375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:13:58.894956 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-5d8def8a-b8d2-4693-8788-eeae516abd00 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1266 microversion: 2.1 time: 0.148120 Jul 24 01:13:58.895465 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 531/1069] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:58 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:13:59.913334 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:13:59.914040 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:13:59.920312 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:59.920762 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:13:59.921531 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:13:59.921968 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:14:00.055446 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:14:00.063104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-0db8a81f-9e32-4c10-909f-1ab0dd720631 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1453 microversion: 2.1 time: 0.154920 Jul 24 01:14:00.063684 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 539/1070] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:13:59 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1453 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:14:00.080424 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:14:00.081032 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:14:00.087978 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:14:00.088421 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:14:00.089206 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:14:00.089639 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:14:00.217398 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:14:00.225131 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-10c953d2-9fca-441f-b341-364b8790adef tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1453 microversion: 2.1 time: 0.148518 Jul 24 01:14:00.225700 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 532/1071] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:14:00 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1453 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:14:01.769104 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-f70bba84-9455-427d-aeb0-e920881f9b67 req-d43d11a6-70c8-44e1-918c-f6bbf8d4eb7f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "027a7e7d-2e80-453f-8b90-341fb9877ce8", "tag": "a628883e-1113-4579-bb10-1d2ff4bb2dc2"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:14:01.778489 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-f70bba84-9455-427d-aeb0-e920881f9b67 req-d43d11a6-70c8-44e1-918c-f6bbf8d4eb7f service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:14:01.778960 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-f70bba84-9455-427d-aeb0-e920881f9b67 req-d43d11a6-70c8-44e1-918c-f6bbf8d4eb7f service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:14:01.790519 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-f70bba84-9455-427d-aeb0-e920881f9b67 req-d43d11a6-70c8-44e1-918c-f6bbf8d4eb7f service nova] Creating event network-changed:a628883e-1113-4579-bb10-1d2ff4bb2dc2 for instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 on hv-cinder-maste Jul 24 01:14:01.795644 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-f70bba84-9455-427d-aeb0-e920881f9b67 req-d43d11a6-70c8-44e1-918c-f6bbf8d4eb7f service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031038 Jul 24 01:14:01.796239 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 540/1072] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:14:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:15:29.557201 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:15:29.557846 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:29.558216 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:29.558500 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG dbcounter [-] [117956] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117956) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:15:29.561601 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_api:SELECT=30 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:15:29.562355 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG dbcounter [-] [117955] Writing DB stats nova_cell1:SELECT=70 {{(pid=117955) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 24 01:15:29.563020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-ed3b47f0-c392-4099-b749-acb11721285b req-ed4339b2-945c-4a9e-9049-3e6fe9097550 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "027a7e7d-2e80-453f-8b90-341fb9877ce8", "tag": "a628883e-1113-4579-bb10-1d2ff4bb2dc2"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:15:29.567346 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:29.568437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:29.569238 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:29.569763 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:29.574289 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ed3b47f0-c392-4099-b749-acb11721285b req-ed4339b2-945c-4a9e-9049-3e6fe9097550 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:29.574741 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ed3b47f0-c392-4099-b749-acb11721285b req-ed4339b2-945c-4a9e-9049-3e6fe9097550 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:29.587110 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-ed3b47f0-c392-4099-b749-acb11721285b req-ed4339b2-945c-4a9e-9049-3e6fe9097550 service nova] Creating event network-changed:a628883e-1113-4579-bb10-1d2ff4bb2dc2 for instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 on hv-cinder-maste Jul 24 01:15:29.593358 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-ed3b47f0-c392-4099-b749-acb11721285b req-ed4339b2-945c-4a9e-9049-3e6fe9097550 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038920 Jul 24 01:15:29.597036 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 541/1073] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:15:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:15:29.597979 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:15:29.683688 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-e92ff009-0286-4dd1-88c7-b511c536fbdf tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "DELETE /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 204 len: 0 microversion: 2.1 time: 0.131143 Jul 24 01:15:29.685778 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 533/1074] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:15:29 2022] DELETE /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:15:29.693740 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:29.694571 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:29.703073 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:29.703605 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:29.704454 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:29.705010 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:30.104083 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:15:30.111551 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-ab36abd4-dd90-409b-a278-0169b0ce528c tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1578 microversion: 2.1 time: 0.422470 Jul 24 01:15:30.112125 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 542/1075] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:29 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1578 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 24 01:15:31.130330 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:31.130963 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:31.138336 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:31.138796 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:31.139570 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:31.140001 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:31.296869 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:15:31.304748 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-c97e826b-37b0-4d32-9879-847d98fd7a2e tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1578 microversion: 2.1 time: 0.178701 Jul 24 01:15:31.305324 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 534/1076] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:31 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1578 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:15:31.818747 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-ffdd6f75-24f8-4b20-ac27-34e92a34889d req-7b9d39a1-8205-4bdf-b5b5-e2c1a5964686 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "027a7e7d-2e80-453f-8b90-341fb9877ce8", "name": "network-vif-deleted", "tag": "a628883e-1113-4579-bb10-1d2ff4bb2dc2"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:15:31.828115 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ffdd6f75-24f8-4b20-ac27-34e92a34889d req-7b9d39a1-8205-4bdf-b5b5-e2c1a5964686 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:31.828584 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ffdd6f75-24f8-4b20-ac27-34e92a34889d req-7b9d39a1-8205-4bdf-b5b5-e2c1a5964686 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:31.841956 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-ffdd6f75-24f8-4b20-ac27-34e92a34889d req-7b9d39a1-8205-4bdf-b5b5-e2c1a5964686 service nova] Creating event network-vif-deleted:a628883e-1113-4579-bb10-1d2ff4bb2dc2 for instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 on hv-cinder-maste Jul 24 01:15:31.846915 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-ffdd6f75-24f8-4b20-ac27-34e92a34889d req-7b9d39a1-8205-4bdf-b5b5-e2c1a5964686 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031801 Jul 24 01:15:31.847509 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 543/1077] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:15:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:15:32.321825 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:32.322487 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:32.328964 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:32.329422 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:32.330194 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:32.330636 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:32.370427 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:32.370886 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:32.371343 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.objects.instance [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lazy-loading 'fault' on Instance uuid 027a7e7d-2e80-453f-8b90-341fb9877ce8 {{(pid=117955) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 24 01:15:32.411442 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:15:32.419833 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-813b940c-1321-4ce5-b042-c85db5db1a11 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 200 len: 1230 microversion: 2.1 time: 0.102259 Jul 24 01:15:32.420409 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 535/1078] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:32 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 1230 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:15:33.438402 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:33.439156 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 027a7e7d-2e80-453f-8b90-341fb9877ce8] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:33.446006 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:33.446705 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:33.447483 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:33.447912 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:33.461465 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] HTTP exception thrown: Instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 could not be found. Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Request method failure captured: Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 HTTP/1.1 Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jue5UKCAbGiAW1r_RNzRV6ogCQZtRChLqeUbe9Cpa8bYw2kmotTJdDbgbfZJcnFoD9f_MC_J7yN0_O1_NGx1dFQjp3QcqPxfaDCOFP-F9EIyXaoStD7sCDz_9W03UDH4HRNiaf2ZcSmc36yPN2YGFVnWWZlqhpC5VzJp5YT8M2FnWg Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:15:33.463173 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_be64d6ea0f0149608c9a823425a0e811", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 7c545f7be7794566b639c29c0717c17a Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 could not be found. Jul 24 01:15:33.464868 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '027a7e7d-2e80-453f-8b90-341fb9877ce8'} Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 could not be found. Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:15:33.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 could not be found. Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 could not be found. Jul 24 01:15:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:33.468706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Returning 404 to user: Instance 027a7e7d-2e80-453f-8b90-341fb9877ce8 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:15:33.468706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-2827afca-2968-4198-9ea1-580d8c763602 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8" status: 404 len: 111 microversion: 2.1 time: 0.030800 Jul 24 01:15:33.468706 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 544/1079] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:33 2022] GET /compute/v2.1/servers/027a7e7d-2e80-453f-8b90-341fb9877ce8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:15:36.512349 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:36.512927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:36.514865 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-ea1ac37d-f90d-493e-bb24-bd8ff12657cc req-ea8204cc-5fb7-48ff-9cec-d09eaa28209c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "495ea0b1-09f6-45d1-9dc1-f47bd463db92", "tag": "a808a619-ba7a-4d1d-946b-70726603a48f"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:15:36.519730 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:36.520245 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:36.521280 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:36.521785 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:36.524950 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ea1ac37d-f90d-493e-bb24-bd8ff12657cc req-ea8204cc-5fb7-48ff-9cec-d09eaa28209c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:36.525628 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-ea1ac37d-f90d-493e-bb24-bd8ff12657cc req-ea8204cc-5fb7-48ff-9cec-d09eaa28209c service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:36.537492 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-ea1ac37d-f90d-493e-bb24-bd8ff12657cc req-ea8204cc-5fb7-48ff-9cec-d09eaa28209c service nova] Creating event network-changed:a808a619-ba7a-4d1d-946b-70726603a48f for instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 on hv-cinder-maste Jul 24 01:15:36.544497 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-ea1ac37d-f90d-493e-bb24-bd8ff12657cc req-ea8204cc-5fb7-48ff-9cec-d09eaa28209c service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033346 Jul 24 01:15:36.545401 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 545/1080] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:15:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 24 01:15:36.550710 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Going to try to terminate instance {{(pid=117955) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 24 01:15:36.641517 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-7526944f-a9db-47df-b5e1-8fb54027e0d4 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "DELETE /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 204 len: 0 microversion: 2.1 time: 0.133469 Jul 24 01:15:36.642484 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 536/1081] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:15:36 2022] DELETE /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 24 01:15:36.652331 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:36.653029 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:36.659536 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:36.659989 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:36.660819 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:36.661261 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:36.799510 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.policy [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117956) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:15:36.807290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-eae8d3e7-dfe0-4b2d-88c2-92b18a5d9283 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1579 microversion: 2.1 time: 0.160307 Jul 24 01:15:36.807863 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 546/1082] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:36 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1579 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:15:37.825994 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:37.827324 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:37.833912 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:37.834362 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:37.835157 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:37.836161 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:37.966173 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:15:37.973775 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-985245e0-3db7-40f4-818f-b6c131866f9b tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1579 microversion: 2.1 time: 0.152243 Jul 24 01:15:37.974566 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 537/1083] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:37 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1579 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:15:38.719396 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [req-35e2955b-f53e-431d-b505-4988ab7e2315 req-e5d2b60d-eac6-4f31-ab56-f88824c29b49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "495ea0b1-09f6-45d1-9dc1-f47bd463db92", "name": "network-vif-deleted", "tag": "a808a619-ba7a-4d1d-946b-70726603a48f"}]} {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 24 01:15:38.729112 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-35e2955b-f53e-431d-b505-4988ab7e2315 req-e5d2b60d-eac6-4f31-ab56-f88824c29b49 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:38.729564 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [req-35e2955b-f53e-431d-b505-4988ab7e2315 req-e5d2b60d-eac6-4f31-ab56-f88824c29b49 service nova] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:38.740785 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.compute.server_external_events [req-35e2955b-f53e-431d-b505-4988ab7e2315 req-e5d2b60d-eac6-4f31-ab56-f88824c29b49 service nova] Creating event network-vif-deleted:a808a619-ba7a-4d1d-946b-70726603a48f for instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 on hv-cinder-maste Jul 24 01:15:38.746389 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [req-35e2955b-f53e-431d-b505-4988ab7e2315 req-e5d2b60d-eac6-4f31-ab56-f88824c29b49 service nova] 10.222.0.18 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030724 Jul 24 01:15:38.747020 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 547/1084] 10.222.0.18 () {64 vars in 1317 bytes} [Sun Jul 24 01:15:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 24 01:15:38.991927 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:38.992541 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:38.999901 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:39.000578 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:39.001546 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:39.002154 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:39.067533 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.policy [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7c545f7be7794566b639c29c0717c17a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be64d6ea0f0149608c9a823425a0e811', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117955) authorize /opt/stack/nova/nova/policy.py:199}} Jul 24 01:15:39.077705 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-0b33b070-655e-4d6f-9fff-78bf911dd684 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 200 len: 1225 microversion: 2.1 time: 0.090026 Jul 24 01:15:39.078293 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 538/1085] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:38 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 24 01:15:40.097814 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:40.098513 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 495ea0b1-09f6-45d1-9dc1-f47bd463db92] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:40.104871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:40.105320 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:40.106082 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:40.106520 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:40.119797 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] HTTP exception thrown: Instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 could not be found. Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Request method failure captured: Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 HTTP/1.1 Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jue5UKCAbGiAW1r_RNzRV6ogCQZtRChLqeUbe9Cpa8bYw2kmotTJdDbgbfZJcnFoD9f_MC_J7yN0_O1_NGx1dFQjp3QcqPxfaDCOFP-F9EIyXaoStD7sCDz_9W03UDH4HRNiaf2ZcSmc36yPN2YGFVnWWZlqhpC5VzJp5YT8M2FnWg Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:15:40.121400 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_be64d6ea0f0149608c9a823425a0e811", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 7c545f7be7794566b639c29c0717c17a Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 could not be found. Jul 24 01:15:40.123088 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '495ea0b1-09f6-45d1-9dc1-f47bd463db92'} Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 could not be found. Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:15:40.124160 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 could not be found. Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 could not be found. Jul 24 01:15:40.125852 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.127461 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Returning 404 to user: Instance 495ea0b1-09f6-45d1-9dc1-f47bd463db92 could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:15:40.127461 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-df3cfb76-1720-42ba-b419-e8e8b264347a tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92" status: 404 len: 111 microversion: 2.1 time: 0.030843 Jul 24 01:15:40.127461 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 548/1086] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:40 2022] GET /compute/v2.1/servers/495ea0b1-09f6-45d1-9dc1-f47bd463db92 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:15:40.559629 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:40.560165 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.compute.api [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117955) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:40.566748 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:40.567229 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:40.567998 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:40.568437 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG oslo_concurrency.lockutils [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117955) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:40.582357 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.wsgi [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] HTTP exception thrown: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Request method failure captured: Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: request: DELETE /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d HTTP/1.1 Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept: application/json Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Accept-Encoding: identity Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Connection: close Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Length: 0 Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Content-Type: application/json Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Host: 10.222.0.18 Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Openstack-System-Scope: None Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: User-Agent: python-urllib3/1.26.9 Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Auth-Token: gAAAAABi3Jue5UKCAbGiAW1r_RNzRV6ogCQZtRChLqeUbe9Cpa8bYw2kmotTJdDbgbfZJcnFoD9f_MC_J7yN0_O1_NGx1dFQjp3QcqPxfaDCOFP-F9EIyXaoStD7sCDz_9W03UDH4HRNiaf2ZcSmc36yPN2YGFVnWWZlqhpC5VzJp5YT8M2FnWg Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Id: None Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Domain-Name: None Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Identity-Status: Confirmed Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Is-Admin-Project: True Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Id: default Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Domain-Name: Default Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Project-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Role: reader,member Jul 24 01:15:40.584053 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Roles: reader,member Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_be64d6ea0f0149608c9a823425a0e811", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-Tenant-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Id: default Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Domain-Name: Default Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Id: 7c545f7be7794566b639c29c0717c17a Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: X-User-Name: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: method: > Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: exception: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.585610 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: action_args: {'id': '863bc791-b8b0-4b49-bf36-6dd4ab8b039d'} Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:15:40.587375 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.588878 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.590544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Returning 404 to user: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. {{(pid=117955) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:15:40.590544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-636a2473-2b38-4217-be80-367961ff08cd tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "DELETE /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 404 len: 111 microversion: 2.1 time: 0.029925 Jul 24 01:15:40.590544 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 539/1087] 10.222.0.18 () {58 vars in 1238 bytes} [Sun Jul 24 01:15:40 2022] DELETE /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:15:40.594123 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '>' {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:40.594735 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.compute.api [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [instance: 863bc791-b8b0-4b49-bf36-6dd4ab8b039d] Fetching instance by UUID {{(pid=117956) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 24 01:15:40.600871 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:40.601395 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:40.602334 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 24 01:15:40.602786 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG oslo_concurrency.lockutils [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Lock "a1472727-7c57-4e7e-873f-6498708782b3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117956) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 24 01:15:40.616339 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.wsgi [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] HTTP exception thrown: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Request method failure captured: Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: request: GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d HTTP/1.1 Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept: application/json Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Accept-Encoding: identity Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Connection: close Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Length: 0 Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Content-Type: application/json Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Host: 10.222.0.18 Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Openstack-System-Scope: None Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: User-Agent: python-urllib3/1.26.9 Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Auth-Token: gAAAAABi3Jue5UKCAbGiAW1r_RNzRV6ogCQZtRChLqeUbe9Cpa8bYw2kmotTJdDbgbfZJcnFoD9f_MC_J7yN0_O1_NGx1dFQjp3QcqPxfaDCOFP-F9EIyXaoStD7sCDz_9W03UDH4HRNiaf2ZcSmc36yPN2YGFVnWWZlqhpC5VzJp5YT8M2FnWg Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Id: None Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Domain-Name: None Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Identity-Status: Confirmed Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Is-Admin-Project: True Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Id: default Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Domain-Name: Default Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Project-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Role: reader,member Jul 24 01:15:40.618586 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Roles: reader,member Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/volume/v3/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2/be64d6ea0f0149608c9a823425a0e811"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18:8080/v1/AUTH_be64d6ea0f0149608c9a823425a0e811", "adminURL": "http://10.222.0.18:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.18/compute/v2.1"}]}] Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Id: be64d6ea0f0149608c9a823425a0e811 Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-Tenant-Name: tempest-TestVolumeBootPattern-634892198 Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Id: default Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Domain-Name: Default Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Id: 7c545f7be7794566b639c29c0717c17a Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: X-User-Name: tempest-TestVolumeBootPattern-634892198-project Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: method: > Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: exception: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.620247 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: action_args: {'id': '863bc791-b8b0-4b49-bf36-6dd4ab8b039d'} Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: {{(pid=117956) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 24 01:15:40.621063 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. Jul 24 01:15:40.622290 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: ERROR nova.api.openstack.wsgi Jul 24 01:15:40.623410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: DEBUG nova.api.openstack.wsgi [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Returning 404 to user: Instance 863bc791-b8b0-4b49-bf36-6dd4ab8b039d could not be found. {{(pid=117956) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 24 01:15:40.623410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO nova.api.openstack.requestlog [None req-575f7c36-e6ed-4758-9d85-344a698667d7 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d" status: 404 len: 111 microversion: 2.1 time: 0.029640 Jul 24 01:15:40.623410 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: [pid: 117956|app: 0|req: 549/1088] 10.222.0.18 () {58 vars in 1235 bytes} [Sun Jul 24 01:15:40 2022] GET /compute/v2.1/servers/863bc791-b8b0-4b49-bf36-6dd4ab8b039d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 24 01:15:41.911055 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: DEBUG nova.api.openstack.wsgi [None req-f8504612-c01a-4ffc-ae86-b77f782abd87 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] Calling method '.version_select at 0x7f5e303ad160>' {{(pid=117955) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 24 01:15:41.914888 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: ERROR oslo.messaging._drivers.impl_rabbit [None req-f8504612-c01a-4ffc-ae86-b77f782abd87 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [d422fd99-2a64-48fa-8802-603de0ccf148] AMQP server on 10.222.0.18:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 24 01:15:42.922454 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [None req-f8504612-c01a-4ffc-ae86-b77f782abd87 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] [d422fd99-2a64-48fa-8802-603de0ccf148] Reconnected to AMQP server on 10.222.0.18:5672 via [amqp] client with port 38982. Jul 24 01:15:42.941011 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO nova.api.openstack.requestlog [None req-f8504612-c01a-4ffc-ae86-b77f782abd87 tempest-TestVolumeBootPattern-634892198 tempest-TestVolumeBootPattern-634892198-project] 10.222.0.18 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1886212345" status: 202 len: 0 microversion: 2.1 time: 1.033798 Jul 24 01:15:42.941583 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: [pid: 117955|app: 0|req: 540/1089] 10.222.0.18 () {58 vars in 1262 bytes} [Sun Jul 24 01:15:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1886212345 => generated 0 bytes in 1035 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 24 01:16:01.015266 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:16:01.015832 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Sun Jul 24 01:16:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:17:39.457284 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:17:39.457746 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:17:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:19:06.288315 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:19:06.288897 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Sun Jul 24 01:19:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:19:21.022976 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:19:21.023671 dv-cinder-master-3-iscsi devstack@n-api.service[117956]: Sun Jul 24 01:19:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:20:44.686510 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:20:44.687197 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:20:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 24 01:20:59.465277 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 24 01:20:59.465930 dv-cinder-master-3-iscsi devstack@n-api.service[117955]: Sun Jul 24 01:20:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!